Tenerife node
SONG technical management log page

Insert new event related to the Tenerife node!
Add to tasks?
Should someone look more into this or is it fixed?
Date of event
Description
Person(s)
Hardware group
Action performed
Attachment
[Maximum size = 20MB]
Notify
Send out notification e-mail?

The latest 50 events are displayed!
Search the log:
DateDescriptionPerson(s)Hardware GroupAction performedAttachmentEdit
2017-07-26 Dome not moving from around 23:20 UTC - vignetting Mads Dome None - it was not detected. Show Edit
2017-07-24 ThAr bulb replaced Mads Spectrograph A new ThAr bulb replaced the old on. New one has serial number: HMC0304. The number of lines was clearly different. None Edit
2017-07-22 VM song.sscentral.prv was h├Žngning sinece around 11 o'clock yesterday meaning the conductor did not run the whole night. Mads None Restarted song.sscentral.prv and started the conductor again None Edit
2017-07-18 Missing light in coude and vignetting of pupil Mads Telescope No light went to coude at startup. I found stars though. Pointing model (PM) was of. Moving M3 a couple of times reduced the offsets but not enough. I manully added some mean offsets to the observing scripts until I have time to do a full PM. Vignetting of pupil might be plastic cover between Blue Spectrograph and Coude path. None Edit
2017-07-18 No light to coude Mads Telescope M3 was not located correctly. Attached is before and after rotation of M3 the pupil image. Show Edit
2017-07-17 No light to coude Frank and Mads Telescope Tried several things and nothing heleped. We decided to close down at 00 UTC after hours of attempts to fix it. None Edit
2017-07-10 Telescope was below 16 degrees (which was okay). The monitor wanted to park the telescope but an observation started in the middle of the movement and telescope never got to park position. Mads None Code needs to be corrected None Edit
2017-07-10 Dome not aligned Frank Dome at ut=05:30 noticed dome was not aligned. Powercycled the dome controller ... this worked. None Edit
2017-07-08 Dome parked in wrong position Frank Dome Rebooted the dome controller and cleared all errors from Astelos manually. Will hopefully start up correctly on its own tonight. None Edit
2017-07-07 At sunset the both limit switch active error happened Mads Telescope Tried to do the things I could but it did not help. Skyped with Michael Ruder who helped out. None Edit
2017-07-07 scratch had only 288GB left of disk space Mads Servers Removed all data and extracted data up to the 1/7 2017 including skycam images which has provided 2.9TB free disk space. None Edit
2017-07-06 Test of blue spectrograph from 22 to 00 UTC Simon Blue spectrograph testing None Edit
2017-07-04 Errors were not cleared after power cycle of dome controller. Mads Telescope and Dome Cleared the two errors and restarted the monitor to allow observations right away. None Edit
2017-06-29 Dome flap open and pointinge ~50deg. from the wind direction with ~13-15m/s .. Frank Dome and flap Discovered this at sun_alt~-6 so closed the mirror covers with Astelos, but waited for the monitor to close down as normal (which worked fine). Show Edit
2017-06-28 Telescope was stationary after 4:20 UTC Mads Telescope went a bit below the software limit of 16 degrees which made it "stuck". Corrections to the code has been made and the telescope should be parked when this occurs in the future. None Edit
2017-06-18 Power failure at 21:35 UTC. Mads All This was not detected before the following morning. No SMS was sent out. The night operator was asked to check and he saw the DIF 3 relay was switched off. Show Edit
2017-06-15 Dome not aligned for the last few hours of the night. Frank Dome Manually moved the dome +/-50 deg in offset with Astelos - at some point the were 'serious errors' in astelos which I acknowledged ... after ~1 minute setting the offset to zero cured the problem. None Edit
2017-06-13 Scratch not mounted om kvmtenerife and vms Mads and Frank Servers Mounted scratch manually "mount -a" as root. This was an issue occurring after the power outage. None Edit
2017-06-12 Replacement stelco rack batteries Antonio Rack STELCO The tape in the photo is because it came broken, but does not lose acid. And the voltage is fine. (To lighten the change, left the wires of the previous battery is so the broken was seen.) Show Edit
2017-06-07 A disk for SODA was full and SODA was down. Mads, Rasmus, Ditte, Eric Servers Eric and Rasmus worked on it bur restoring from backup was slow and SODA was down during night which caused the conductor to fail obviously. None Edit
2017-06-05 Dome was offset again Mads Dome Power cycled the dome controller Show Edit
2017-05-29 Stellas dust sensor stopped working Frank Dust-sensor Disabled the dust-sensor as the weather looks great. None Edit
2017-05-27 Monitor hangs Frank Telescope/computers restarted tcs_comm and monitor None Edit
2017-05-25 Dome not aligned Frank Dome around 21:10 UT I noticed a slight vignetting of the pupil for Muher ... did various offsets with Astelos (up to +/-25deg) ... after several attempts a dome offset of 0 again provided a fully illuminated pupil.... None Edit
2017-05-24 Dome was not aligned since 21:30 UTC Mads Dome Dome controller was finally power cycled automatically but errors not cleared. I manually cleared thorugh AsTelOS and all was good again. None Edit
2017-05-23 M8 positions were not perfect for halo, thar and blue sky Mads Spectrograph These were corrected for all slits. None Edit
2017-05-23 Morning: Pointing model was not perfect any more Mads Telescope Made a new one with 62 measurements and a RMS of 1.1 arcseconds None Edit
2017-05-22 A general power failure happened just before 16:00 UTC. Mads All All was back to normal, I think, around 18:00 UTC None Edit
2017-05-20 Morning: The RCD triggered at power on of the telescope before sunset after bad weather. Mads Telescope and electrical installation Later Rafa removed the RCD from the circuit and the next night went without problems. None Edit
2017-05-19 A power consumption counter was installed at the power line entrance in the container. Power cables were reconnected and all cables including power to the telescope was send through a RCD (Recidual Current Device) which was triggered at telescope power on causing power outages to the telescope. Mads Telescope and electrical installation Next day Miguel changed the RCD to a new one and all seemed to work again. None Edit
2017-05-19 Shut down in the morning did not execute correctly Mads Telescope and Dome Everything seemed fine but the dome was a bit off. Rotated the dome using offsets and it was correctly alligned again and I powered off. None Edit
2017-05-17 An AO error occurred and was not acknowledged because of the UPS warning properly Mads Telescope Cleared through AsTelOS and restarted monitor to force a quick re-open. Worked Show Edit
2017-05-17 Webserver/web pages not responding Mads Servers Hard reboot from sscentral of srf got it running again None Edit
2017-05-16 Dome not behaving well Mads Dome Monitor detected the misbehaving dome and made a power cycle. The errors were not acknowledged and a close down started. Because of errors the shutdown failed and sms was send. This has happened several times. None Edit
2017-04-24 Weather-station reported constant values for all parameters - but these were within the allowed limits, so observing continued this morning, even though the humiidity was above the limit (did not seem to be raining though). Frank Weatherstation Stopped the weather service, waited 20sec, restarted the weatherservide. This made the station report proper values and the the telescope is closing now. None Edit
2017-04-18 songwriter failed Frank Data reduction None - because of the failure yesterday for sstenerife, no calibration files were available, so songwriter failed. We need to fix the reduction manually at some point. None Edit
2017-04-13 Failing OR script Mads None Corrections to the code was made to check for wrong focus. The script was trying to make a string number to an integer... failing... this was causing the conductor to - correctly - insert ORs... leading to more than 200 inserted ORs which all failed until I fixed the bug just before 1 UTC. All failed ORs inserted by the conductor was then deleted from the database... None Edit
2017-04-12 Telescope out of focus Mads Telescope The position was not set for the telescope focus at each repoint meaning we were of by about 1.8mm each time. None Edit
2017-04-12 Conductor test night Mads None The conductor was tested and some errors in the code was corrected at 2:30 UTC. None Edit
2017-04-10 Disk space on scratch was running low Mads Servers Removed old data up until 2017-01-01 from star_spec and up to 2017-04-01 of skycam1 images. This made 1.7 TB of free disk space. None Edit
2017-04-04 Stellas dust sensor stopped working Frank Dust-sensor Disabled the dust-sensor as the weather looks great. None Edit
2017-04-03 Monitor was hanging when trying to open mirror covers Mads Telescope There were a bug in the new way the TSI now speeks to our software which was causing the command to never end. This was fixed. None Edit
2017-04-02 automatic focus failed -- focus reported 6mm !! (12mm offset and value=-6) Astelos/TSI could not recover. Frank Telescope focus Stopped or_scheduler and monitor - telescope closed power off. Restart monitor, Things started normally but it took a LONG time before focus was found - first object did not reach focus in 120s, second object needed ~3min before finding the correct focus. None Edit
2017-03-30 At: 2017-03-30 22:38 UTC: Telescope software lost connection to TSI - socket timeout Mads, Simon, Christoffer Telescope Simon and Christoffer restarted the tcs_comm_daemon which re-astablished the connection None Edit
2017-03-29 Time on luckycam1 and luckycam2 were not corrected using ntpdate Mads servers Added to the crontab to update time using: ntpdate 192.168.66.1 None Edit
2017-03-28 At: 2017-03-29 00:40 UTC : The connection to the dome controller was lost and the error could not be cleared. Mads Dome Looked at the problem and the dome would not move. Power cycled the dome controller and afterwards all seemed fine. Show Edit
2017-03-23 Observed occultation of star TYC-647-586-1, with lucky cams in conv. mode. there seem to be a ~5min. time offset in the red images, .... Frank Lucky Imaging None None Edit
2017-03-23 Dome and telescope was completely mis-aligned after observing Ceres occulation and starting spectroscopic observations. Frank Dome Rebooted the dome controller None Edit
2017-03-21 At 23:40 UTC an error about the active optics was raised and the telescope was shut down Mads Telescope Started Astelos to look at the problem. The error was then cleared when everything looked okay. Show Edit
2017-03-21 M3 - both limit switches active - problem Mads Telescope Skype message send to Michael Ruder and Astelco and Tau-Tec notified by e-mail. Error left unresolved for them to have a look before any action will be initiated. None Edit