Time Sync with external server

Requests for new functions or software apps
Post Reply
rimai
Posts: 12881
Joined: Fri Mar 18, 2011 6:47 pm

Time Sync with external server

Post by rimai »

I was asked about the possibility of time sync with an external server and decided to give it a shot.
It's currently being synced with reefangel.com server, which is in Eastern Daylight Time (EDT) at the moment.
This option adds just a little less memory space than the menu based date/time change implemented by Curt.
Let me know if it works accordingly.

Code: Select all

const prog_char SyncGET[] PROGMEM = "GET /status/time.aspx\n\n";

void SyncTime(int offset)
{
  PROGMEMprint(SyncGET);
  time_t SyncTimeout=millis();
  long SyncedTime=0;
  boolean tData=false;
  while (millis()-SyncTimeout<1000)
  {
	#if defined(wdt) || defined(forcewdt)
	wdt_reset();
	#endif
    byte inStr=Serial.read();
    if (inStr==0x0d && tData)
    {
      SyncTimeout=0;
      break;
    }
    if (inStr>='0' && inStr<='9' && tData)
    {
      Serial.println(inStr,DEC);
      SyncedTime*=10;
      SyncedTime+=inStr-'0';
    }
    if (inStr==93) tData=true;
  }
  if (SyncTimeout==0)
  {
    if (SyncedTime>1310920000)
    {
	  SyncedTime+=(offset*3600);
      setTime(SyncedTime);  // Unix time
      now();
      ReefAngel.RTC.set(now());
    }
    //ReefAngel.LCD.DrawText(0,255,10,10, SyncedTime);
  }
}
I pushed to my github too, if you just want to grab it that way.
You will need to make sure that your wifi is setup properly with these commands:

Code: Select all

set u m 2
set i p 18
set d n www.reefangel.com
set i h 198.171.134.6
set i r 80
Roberto.
Paul Viscovich
Posts: 26
Joined: Mon Jul 18, 2011 8:06 pm
Location: Dublin Ca
Contact:

Re: Time Sync with external server

Post by Paul Viscovich »

Thanks much i like the solution..

Next question and i will assume that the anwser is no but can you point the wifi at two diffrent sites

And if not would there be a way to adjust the time on your site or at least the returned time by say passing a -8 or -7 to it for pst.. in the get statement so the aspx code could return a offset from your server time

and maybe -8x or -8x to ignore DST changes so AZ could send a -7x and the time could be caculated to be correct..

The real problem in the time keeping is not so much the acuracy as in the DST changes that cause the lights to come on and off 1 hour wrong.
http://forum.reefangel.com/status/banner_3.aspx?id=Paul Viscovich&t=-7

14 Gallon Bio Cube with INTANK filtration Thanks Diablo Corals
rimai
Posts: 12881
Joined: Fri Mar 18, 2011 6:47 pm

Re: Time Sync with external server

Post by rimai »

Although there is a way to get the wifi to point to different servers, you would have to change it each time you would want to connect to that particular server. I'd not recommend doing so, but it is possible.
The offset is an input of the function itself, which would be always related to server time and not Greenwich time. Since the time on the server updates with daylight savings, you would be able to get it updated as soon as the server updates itself.
For example, to get time on PDT, simple use SyncTime(-3).
Roberto.
Paul Viscovich
Posts: 26
Joined: Mon Jul 18, 2011 8:06 pm
Location: Dublin Ca
Contact:

Re: Time Sync with external server

Post by Paul Viscovich »

Ok should have read the code better the first time... I see that.. what about areas that do not use DST and are on ST all the time..

when your server changed they would show the wrong time..
http://forum.reefangel.com/status/banner_3.aspx?id=Paul Viscovich&t=-7

14 Gallon Bio Cube with INTANK filtration Thanks Diablo Corals
rimai
Posts: 12881
Joined: Fri Mar 18, 2011 6:47 pm

Re: Time Sync with external server

Post by rimai »

I thought of that too, but the problem is that if we use GMT, we'll never have the DST calculated. Any suggestion?
PDT is GMT-7 and PST is GMT-8, right?
Roberto.
Paul Viscovich
Posts: 26
Joined: Mon Jul 18, 2011 8:06 pm
Location: Dublin Ca
Contact:

Re: Time Sync with external server

Post by Paul Viscovich »

Of cource if you were in AZ you could just not use the code bacuse the closk is prety stable and the real problem is the DST zones

correct So my idea would not work... :(

The only way would be to maybe use the same idea that windows does and keep a xref on the server for look up for the dst dates

UTC

If the time is in UTC, add a "Z" directly after the time without a space. "Z" is the zone designator for the zero UTC offset. "09:30 UTC" is therefore represented as "09:30Z" or "0930Z". "14:45:15 UTC" would be "14:45:15Z" or "144515Z".

UTC time is also known as "Zulu" time, since "Zulu" is the ICAO spelling alphabet code word for "Z".

[edit] Time zone as offsets from UTC

Time zone are written as offset from UTC in the format ±[hh]:[mm], ±[hh][mm], or ±[hh]. So if the time being described is one hour ahead of UTC (such as the time in Berlin during the winter), the zone designator would be "+01:00", "+0100", or simply "+01". This is appended to the time in the same way that 'Z' was above. The offset from UTC changes with daylight saving time, e.g. a time offset in Chicago, which is in the North American Central Time Zone, would be "−06:00" for the winter (Central Standard Time) and "−05:00" for the summer (Central Daylight Time).
http://forum.reefangel.com/status/banner_3.aspx?id=Paul Viscovich&t=-7

14 Gallon Bio Cube with INTANK filtration Thanks Diablo Corals
Paul Viscovich
Posts: 26
Joined: Mon Jul 18, 2011 8:06 pm
Location: Dublin Ca
Contact:

Re: Time Sync with external server

Post by Paul Viscovich »

http://www.onlineaspect.com/2007/06/08/ ... avascript/

This is in java script but the site might provide a way to do it by having the conroller post it time to the server and then doing the conversion and then sending back and setting the time
http://forum.reefangel.com/status/banner_3.aspx?id=Paul Viscovich&t=-7

14 Gallon Bio Cube with INTANK filtration Thanks Diablo Corals
Paul Viscovich
Posts: 26
Joined: Mon Jul 18, 2011 8:06 pm
Location: Dublin Ca
Contact:

Re: Time Sync with external server

Post by Paul Viscovich »

https://bitbucket.org/pellepim/jstimezo ... /wiki/Home

this is recomended over the other one
http://forum.reefangel.com/status/banner_3.aspx?id=Paul Viscovich&t=-7

14 Gallon Bio Cube with INTANK filtration Thanks Diablo Corals
rimai
Posts: 12881
Joined: Fri Mar 18, 2011 6:47 pm

Re: Time Sync with external server

Post by rimai »

This solution would not work for us because both versions posted rely on browser javascript actually calculate date in june to compare with january ro verify DST exists or not. The controller doesn't have a way to find that out.
Roberto.
Paul Viscovich
Posts: 26
Joined: Mon Jul 18, 2011 8:06 pm
Location: Dublin Ca
Contact:

Re: Time Sync with external server

Post by Paul Viscovich »

well this solution is good for those areas that do use DST as does the servers TZ so 90% of the world is covered..

and if you dont use dst then your time does not change the 1 hour and you dont have the problem

I have a diffrent idea.

I assume that you can read the time of the server and knowing that the server is in an area that uses DST we can standardize the Time by subing 60 minutes from the server time to get STD time alwasy or if it is standard time then re dont sub the 60 minutes (all done on the server side)

Then we pass the command from the Controller to the server TimeGet (+-OffSetMinutes, DSTOffsetMinutes)

Then the server could take that info and on the standardized time it could set the time by standardized time - offsetminutes - dstoffsetminutes and return a time that should be corect based on the info passed to it.. now if there are diffrent dates the routine would fail if the DST change date is not the same as EST/EDT but it would get many more folks around the world much closer to the correct time.

it all depends if the controller can post its offset and dstoffset

so in california i would sent the gettime (-180,-60)

the server would take the currect time and if dst sub 60 minutes if not set the variable to the time

then subtract the -180 (3 hour time zone diff) and + 60 for the dst

assumeing NY never changes its DST (we have standardized the time to force this condition in software) and california does then we would be 3 hours diffrent then 4 hours diffrent when we changed

hence the -180 and if the server dst flag is set we sub the dst offset as well

God i hope this is making some sence
http://forum.reefangel.com/status/banner_3.aspx?id=Paul Viscovich&t=-7

14 Gallon Bio Cube with INTANK filtration Thanks Diablo Corals
Paul Viscovich
Posts: 26
Joined: Mon Jul 18, 2011 8:06 pm
Location: Dublin Ca
Contact:

Re: Time Sync with external server

Post by Paul Viscovich »

Ok so i had this long post going untill i figured out that the date is a moving date for DST and every year it is changing.. what is needed is a world database that could be used to tell if or when you change then a code could be sent to the server to calculate based on the est rules your server goes by..


Oh boy this is a pain....

like this http://twiki.org/cgi-bin/xtra/tzdatepick.html#GMT
http://forum.reefangel.com/status/banner_3.aspx?id=Paul Viscovich&t=-7

14 Gallon Bio Cube with INTANK filtration Thanks Diablo Corals
Paul Viscovich
Posts: 26
Joined: Mon Jul 18, 2011 8:06 pm
Location: Dublin Ca
Contact:

Re: Time Sync with external server

Post by Paul Viscovich »

Ok new Idea

http://api.geonames.org/timezone?lat=37 ... rname=demo

can you send a string like this from the wifi and then read the responce

or maybe you can get this api for your site

then all you would need to do is pass the lat and lng to it and it would do the rest

you can use just about anything these days to het the long and lat of your location

http://www.geonames.org/export/web-serv ... l#timezone

Timezone
Webservice Type : REST
Url : api.geonames.org/timezone?
Parameters : lat,lng, radius (buffer in km for closest timezone in coastal areas), date (date for sunrise/sunset);
Result : the timezone at the lat/lng with gmt offset (1. January) and dst offset (1. July)
Example http://api.geonames.org/timezone?lat=47 ... rname=demo

This service is also available in JSON format : http://api.geonames.org/timezoneJSON?la ... rname=demo


Element:
timezoneId: name of the timezone (according to olson), this information is sufficient to work with the timezone and defines DST rules, consult the documentation of your development environment.
time: the local current time
rawOffset: the amount of time in hours to add to UTC to get standard time in this time zone. Because this value is not affected by daylight saving time, it is called raw offset.
gmtOffset: offset to GMT at 1. January (deprecated)
dstOffset: offset to GMT at 1. July (deprecated)


<?xml version="1.0" encoding="UTF-8"?>
-<geonames>
-<timezone>
<countryCode>US</countryCode>
<countryName>United States</countryName>
<lat>37.71</lat>
<lng>-121.9</lng>
<timezoneId>America/Los_Angeles</timezoneId>
<dstOffset>-7.0</dstOffset>
<gmtOffset>-8.0</gmtOffset>
<rawOffset>-8.0</rawOffset>
<time>2011-07-19 00:01</time>
<sunrise>2011-07-19 06:00</sunrise>
<sunset>2011-07-19 20:26</sunset>
</timezone>
</geonames>
http://forum.reefangel.com/status/banner_3.aspx?id=Paul Viscovich&t=-7

14 Gallon Bio Cube with INTANK filtration Thanks Diablo Corals
Paul Viscovich
Posts: 26
Joined: Mon Jul 18, 2011 8:06 pm
Location: Dublin Ca
Contact:

Re: Time Sync with external server

Post by Paul Viscovich »

http://api.geonames.org/timezone?lat=37 ... ablocorals

signed up and made an account that means that i can do 33000 time requests a day..

maybe you could set up a free reefangle account

and the

http://api.geonames.org/timezone?lat=37 ... ablocorals

out put could be read back in

Of cource this also could be done server side if teh users cointroller posts to reefangle and the page there posts the data and collects the data and then send the time info back to the controller

Again i may be off base but it is what i have found sofare with out knowing the limmits it is hard to tell what you can do
http://forum.reefangel.com/status/banner_3.aspx?id=Paul Viscovich&t=-7

14 Gallon Bio Cube with INTANK filtration Thanks Diablo Corals
Paul Viscovich
Posts: 26
Joined: Mon Jul 18, 2011 8:06 pm
Location: Dublin Ca
Contact:

Re: Time Sync with external server

Post by Paul Viscovich »

too bad jess needs both of his controllers or i would be working on this from my end ... I cant go wipe his code to play :( maybe one of these days i can buy one for dev work...

Ok enough rambling for one night im off to bed...
http://forum.reefangel.com/status/banner_3.aspx?id=Paul Viscovich&t=-7

14 Gallon Bio Cube with INTANK filtration Thanks Diablo Corals
rimai
Posts: 12881
Joined: Fri Mar 18, 2011 6:47 pm

Re: Time Sync with external server

Post by rimai »

The server can always give me GMT (or if you prefer calling UTC), and also the current time (Eastern Time) with the daylight savings offset that follows the Eastern Time USA daylight savings rules.
But, I actually love that site you refered.
It's got exactly what we need:
http://download.geonames.org/export/dump/timeZones.txt
That XML data also gave me other ideas.
Exact sunset and sunrise hours for a particular area.
So, if one would like to replicate Fiji sunrise and sunset hours, the controller could adjust itself and dim the LED strings according to those hours. Well, this is topic for another thread.
Anyway, I'm going to enter the data into a database and generate an id for each timezone. Then, instead of passing offset to SyncTime function, we pass the id for the timezone to get the correct time adjusted for daylight savings.
Thanks for the research :)
Roberto.
Paul Viscovich
Posts: 26
Joined: Mon Jul 18, 2011 8:06 pm
Location: Dublin Ca
Contact:

Re: Time Sync with external server

Post by Paul Viscovich »

i was thinking the same thing about the sunset sun rise it seems like it could be levraged for many things..
http://forum.reefangel.com/status/banner_3.aspx?id=Paul Viscovich&t=-7

14 Gallon Bio Cube with INTANK filtration Thanks Diablo Corals
rimai
Posts: 12881
Joined: Fri Mar 18, 2011 6:47 pm

Re: Time Sync with external server

Post by rimai »

Ok, so the more I try to come up with a solution, the more I come to realize that it is so much easier to just sync with your computer by using the Client 2.2 twice a year when daylight savings changes.
With the new Client 2.2, you can simply sync it with a click of a button and it can be done through USB or wifi.
Roberto.
binder
Posts: 2871
Joined: Fri Mar 18, 2011 6:20 pm
Location: Illinois
Contact:

Re: Time Sync with external server

Post by binder »

yeah, it is pretty simple and easy to do now that we have the web server responding to the date command.

curt
lukeluke
Posts: 88
Joined: Mon Apr 04, 2011 4:12 am
Location: Rome, Italy

Re: Time Sync with external server

Post by lukeluke »

curt, have you enable this on your .16 version ?
binder
Posts: 2871
Joined: Fri Mar 18, 2011 6:20 pm
Location: Illinois
Contact:

Re: Time Sync with external server

Post by binder »

lukeluke wrote:curt, have you enable this on your .16 version ?
No I did not add in the external time sync functionality to my libraries because of how the discussion went. It's simpler to just either set the date manually or update it via the wifi interface.
I know you do not run Windows (I believe you run Ubuntu linux or some other variant) so you cannot run the client suite. For now, you would have to manually set the date/time yourself via the memory command.

Code: Select all

http://CONTROLLER_IP:2000/dHHMM,MMDD,YY
The format is as follows:
  • HH - hours
  • MM - minutes
  • , - comma
  • MM - month
  • DD - day
  • , - comma
  • YY - year (since 2000)
Everything MUST be in 2 digit format. So single digits (like 8) must have a 0 prefix. Here's an example:

Set the date to August 8th, 2011 at 9:30am

Code: Select all

http://CONTROLLER_IP:2000/d0930,0808,11
The controller will respond with OK and set the date/time on the controller.

NOTE: Type this in the URL box in your web browser to set the date/time.

curt
lukeluke
Posts: 88
Joined: Mon Apr 04, 2011 4:12 am
Location: Rome, Italy

Re: Time Sync with external server

Post by lukeluke »

ok thanks ;-)
lukeluke
Posts: 88
Joined: Mon Apr 04, 2011 4:12 am
Location: Rome, Italy

Re: Time Sync with external server

Post by lukeluke »

curt is possible modify other internal memory parameters trought the browser ?
rimai
Posts: 12881
Joined: Fri Mar 18, 2011 6:47 pm

Re: Time Sync with external server

Post by rimai »

Yes, it is indeed, but the Real Time Clock (RTC) data is not within the internal memory block.
The RTC chip is the one that holds the time and keeps it up.
The controller also syncs with it periodically.
Roberto.
binder
Posts: 2871
Joined: Fri Mar 18, 2011 6:20 pm
Location: Illinois
Contact:

Re: Time Sync with external server

Post by binder »

lukeluke wrote:curt is possible modify other internal memory parameters trought the browser ?
Yes, you most certainly can. It is slightly more complex in doing this though. You must know the location and what type of value is stored there (byte or integer). These locations are stored inside ReefAngel_Globals.h and whether it is a byte or integer.

For working with Bytes:

Code: Select all

// Read the memory value
http://CONTROLLER_IP:2000/mbLOC
// LOC is the memory location

// Write the memory value
http://CONTROLLER_IP:2000/mbLOC,VAL
// there must be a comma (,) between LOC and VAL (which is the value)
For working with Integers:

Code: Select all

// Read the memory value
http://CONTROLLER_IP:2000/miLOC
// LOC is the memory location

// Write the memory value
http://CONTROLLER_IP:2000/miLOC,VAL
// there must be a comma (,) between LOC and VAL (which is the value)
I'm working on a program that will allow you to update the values from it without having to open up a web browser. I'm almost ready for an "alpha" release and hoping to have something later this weekend.

curt
Post Reply