Tenerife node
SONG technical management log page

Insert new event related to the Tenerife node!
Add to tasks?
Should someone look more into this or is it fixed?
Date of event
Hardware group
Action performed
[Maximum size = 20MB]
Send out notification e-mail?

The latest 50 events are displayed!
Search the log:
DateDescriptionPerson(s)Hardware GroupAction performedAttachmentEdit
2017-06-18 Power failure at 21:35 UTC. Mads All This was not detected before the following morning. No SMS was sent out. The night operator was asked to check and he saw the DIF 3 relay was switched off. Show Edit
2017-06-15 Dome not aligned for the last few hours of the night. Frank Dome Manually moved the dome +/-50 deg in offset with Astelos - at some point the were 'serious errors' in astelos which I acknowledged ... after ~1 minute setting the offset to zero cured the problem. None Edit
2017-06-13 Scratch not mounted om kvmtenerife and vms Mads and Frank Servers Mounted scratch manually "mount -a" as root. This was an issue occurring after the power outage. None Edit
2017-06-12 Replacement stelco rack batteries Antonio Rack STELCO The tape in the photo is because it came broken, but does not lose acid. And the voltage is fine. (To lighten the change, left the wires of the previous battery is so the broken was seen.) Show Edit
2017-06-07 A disk for SODA was full and SODA was down. Mads, Rasmus, Ditte, Eric Servers Eric and Rasmus worked on it bur restoring from backup was slow and SODA was down during night which caused the conductor to fail obviously. None Edit
2017-06-05 Dome was offset again Mads Dome Power cycled the dome controller Show Edit
2017-05-29 Stellas dust sensor stopped working Frank Dust-sensor Disabled the dust-sensor as the weather looks great. None Edit
2017-05-27 Monitor hangs Frank Telescope/computers restarted tcs_comm and monitor None Edit
2017-05-25 Dome not aligned Frank Dome around 21:10 UT I noticed a slight vignetting of the pupil for Muher ... did various offsets with Astelos (up to +/-25deg) ... after several attempts a dome offset of 0 again provided a fully illuminated pupil.... None Edit
2017-05-24 Dome was not aligned since 21:30 UTC Mads Dome Dome controller was finally power cycled automatically but errors not cleared. I manually cleared thorugh AsTelOS and all was good again. None Edit
2017-05-23 M8 positions were not perfect for halo, thar and blue sky Mads Spectrograph These were corrected for all slits. None Edit
2017-05-23 Morning: Pointing model was not perfect any more Mads Telescope Made a new one with 62 measurements and a RMS of 1.1 arcseconds None Edit
2017-05-22 A general power failure happened just before 16:00 UTC. Mads All All was back to normal, I think, around 18:00 UTC None Edit
2017-05-20 Morning: The RCD triggered at power on of the telescope before sunset after bad weather. Mads Telescope and electrical installation Later Rafa removed the RCD from the circuit and the next night went without problems. None Edit
2017-05-19 A power consumption counter was installed at the power line entrance in the container. Power cables were reconnected and all cables including power to the telescope was send through a RCD (Recidual Current Device) which was triggered at telescope power on causing power outages to the telescope. Mads Telescope and electrical installation Next day Miguel changed the RCD to a new one and all seemed to work again. None Edit
2017-05-19 Shut down in the morning did not execute correctly Mads Telescope and Dome Everything seemed fine but the dome was a bit off. Rotated the dome using offsets and it was correctly alligned again and I powered off. None Edit
2017-05-17 Webserver/web pages not responding Mads Servers Hard reboot from sscentral of srf got it running again None Edit
2017-05-17 An AO error occurred and was not acknowledged because of the UPS warning properly Mads Telescope Cleared through AsTelOS and restarted monitor to force a quick re-open. Worked Show Edit
2017-05-16 Dome not behaving well Mads Dome Monitor detected the misbehaving dome and made a power cycle. The errors were not acknowledged and a close down started. Because of errors the shutdown failed and sms was send. This has happened several times. None Edit
2017-04-24 Weather-station reported constant values for all parameters - but these were within the allowed limits, so observing continued this morning, even though the humiidity was above the limit (did not seem to be raining though). Frank Weatherstation Stopped the weather service, waited 20sec, restarted the weatherservide. This made the station report proper values and the the telescope is closing now. None Edit
2017-04-18 songwriter failed Frank Data reduction None - because of the failure yesterday for sstenerife, no calibration files were available, so songwriter failed. We need to fix the reduction manually at some point. None Edit
2017-04-13 Failing OR script Mads None Corrections to the code was made to check for wrong focus. The script was trying to make a string number to an integer... failing... this was causing the conductor to - correctly - insert ORs... leading to more than 200 inserted ORs which all failed until I fixed the bug just before 1 UTC. All failed ORs inserted by the conductor was then deleted from the database... None Edit
2017-04-12 Telescope out of focus Mads Telescope The position was not set for the telescope focus at each repoint meaning we were of by about 1.8mm each time. None Edit
2017-04-12 Conductor test night Mads None The conductor was tested and some errors in the code was corrected at 2:30 UTC. None Edit
2017-04-10 Disk space on scratch was running low Mads Servers Removed old data up until 2017-01-01 from star_spec and up to 2017-04-01 of skycam1 images. This made 1.7 TB of free disk space. None Edit
2017-04-04 Stellas dust sensor stopped working Frank Dust-sensor Disabled the dust-sensor as the weather looks great. None Edit
2017-04-03 Monitor was hanging when trying to open mirror covers Mads Telescope There were a bug in the new way the TSI now speeks to our software which was causing the command to never end. This was fixed. None Edit
2017-04-02 automatic focus failed -- focus reported 6mm !! (12mm offset and value=-6) Astelos/TSI could not recover. Frank Telescope focus Stopped or_scheduler and monitor - telescope closed power off. Restart monitor, Things started normally but it took a LONG time before focus was found - first object did not reach focus in 120s, second object needed ~3min before finding the correct focus. None Edit
2017-03-30 At: 2017-03-30 22:38 UTC: Telescope software lost connection to TSI - socket timeout Mads, Simon, Christoffer Telescope Simon and Christoffer restarted the tcs_comm_daemon which re-astablished the connection None Edit
2017-03-29 Time on luckycam1 and luckycam2 were not corrected using ntpdate Mads servers Added to the crontab to update time using: ntpdate None Edit
2017-03-28 At: 2017-03-29 00:40 UTC : The connection to the dome controller was lost and the error could not be cleared. Mads Dome Looked at the problem and the dome would not move. Power cycled the dome controller and afterwards all seemed fine. Show Edit
2017-03-23 Observed occultation of star TYC-647-586-1, with lucky cams in conv. mode. there seem to be a ~5min. time offset in the red images, .... Frank Lucky Imaging None None Edit
2017-03-23 Dome and telescope was completely mis-aligned after observing Ceres occulation and starting spectroscopic observations. Frank Dome Rebooted the dome controller None Edit
2017-03-21 M3 - both limit switches active - problem Mads Telescope Skype message send to Michael Ruder and Astelco and Tau-Tec notified by e-mail. Error left unresolved for them to have a look before any action will be initiated. None Edit
2017-03-21 At 23:40 UTC an error about the active optics was raised and the telescope was shut down Mads Telescope Started Astelos to look at the problem. The error was then cleared when everything looked okay. Show Edit
2017-03-20 Focus somewhat off the correct value - clearly not in focus, two half circles as guide spots. Frank Telescope focus Used Astelos to increase focus by ~.15mm --> this caused the aufofocus routine to get the right focus. Perhpas the initial focus guess is a bit too low ? None Edit
2017-03-08 Database problem. Database had no more diskspace on central Mads, Eric Servers Eric added more diskspace on central in the morning the 9th. None Edit
2017-03-06 Stella Dust service seem to be stopped Frank Dust sensor Disabled the dust-server on HW and stop/started the weather station. Weather seem ok.. None Edit
2017-03-05 Telescope and dome misalignment and other problems Mads Telescope, dome and software It seems the CCC rebooted and the connection was re-established through the TSI. The dome was rotating infinetely and the cure was to power cycle the controller. This is now causing errors for the monitor which has to be handled in a new way, checked and verified. None Edit
2017-03-05 Water cooler for Andor CCD is clogged Mads, Antonio Spectrograph When leaving Mads noticed there were lime and minerals forming in the water cooler but did not have time to do anything. Antonio looked at it and confirmed that it was clogged. None Edit
2017-03-05 Seeing measured in the spectrograph seems too high Mads Telescope and spectrograph The mirrors should be CO2 cleaned as soon as possible to see if this has an effect. None Edit
2017-03-03 LI CCDs were aligned (already from the rough alignment on the 26th). Mads Lucky Imaging Center of field-rotation were determined and a 30 star pointing model were created with an RMS of 0.92" None Edit
2017-03-03 Adjusted coude alignment Mads Spectrograph Moved PST physically again to adjust according to stellar light rather than calibrations. M7 was also adjusted (The pupil rotates with telescope AZ and the center of rotation was centered) None Edit
2017-03-03 Noticed the dome might jump tracks in the dome azimuth rotation when moveing far and the dome gains momentum. Mads, Tau-Tec Dome This is a cause for the misalingment of the dome when it moves around a lot. None Edit
2017-03-03 The new TSI (Telescope Software interface) reports two errors now when the dome controller is power cycled. Before it was just a warning. Mads Telescope and Dome This needs to be handled correctly by the monitor which will now send out an sms and shut down.... most likely a lot more things to come... None Edit
2017-02-28 Calibration light paths were adjusted to hit the slit when M8 was close to idle Mads Spectrograph The fibreholders for ThAr and Sun was adjusted to send the light very close to the slit when M8 in idle (sigu). None Edit
2017-02-28 PST and M8 were adjusted to make M8 and the guider better in the range of M8 movements Mads Spectrograph M8 were shimmed and PST were physically moved a tiny bit None Edit
2017-02-27 Hand paddle for manual control of the telescope and dome was repaired Tau-Tec, Mads Telescope and dome All buttons and print-board was replaced. New location of the paddle is at the bottom of the stairs inside the concrete pier to prevent water to enter. Show Edit
2017-02-26 Astelco and Tau-Tec arrived and will completely renew the software from there side Peter Aniol, Michael Ruder, Mario and Mads Telescope, Dome etc The two guys from Tau-Tec started the upgrade ríght after dinner. The scientific operation will be affected until they are finished and everything is verified and working again. None Edit
2017-02-26 LI CCD alignment Mads Lucky Imaging The small metal things inside of the attachment on RED CCD was rotated 90 degrees which allowed more flexibility in the movement of the CCD. The CCd was moved about 1.5 mm. Show Edit