I have released a new version of my GHCN software. It is much faster at obtaining NOAA data and should work on Mac, Linux or Cygwin on Windows. Download the software here .
Then run these commands from a Linux, Mac or Cygwin command prompt.
tar xvf ghcn.tar cd ghcn_files chmod 755 get ./get cd ghcnd_hcn ./ghcn.exe US.txt > US.csv
This is the software I wrote and use for analyzing the US temperature record. There are tons of command line options. For example, if you want to know what historical temperatures were on March 17 type :
./ghcn.exe US.txt date=0317 > US_0317.csv
As I mentioned, it has a huge range of possibilities for climate analysis, but you will have to dig through the C++ code to figure it out. I will give examples in my posts.
You lost me at “Cygwin”…
Cygwin is a Gnu environment for Windows so you can use it like a UNIX system. Not recommended for non-UNIX users.
Mac and Linux are UNIX based, so they have everything you need already installed.
Thanks Tony, but I’m not brave enough to try it.
BUT if you were to put up a link the the your latest release of “Pulling Back the Curtain” that would be great. I think I didn’t download that last one. And the one I have won’t run anymore I assume because of all the automatic updates that we all get always seems to screw something up.
$ make
g++ -O3 -o ghcn.exe GHCNMain.cpp Main.cpp GHCN.cpp
GHCN.cpp: In function ‘std::__cxx11::string dateString(int, int)’:
GHCN.cpp:267:62: error: ‘sprintf’ was not declared in this scope
sprintf(date, “%s %d”, MONTH_NAMES[month], days_remaining);
^
Makefile:5: recipe for target ‘ghcn.exe’ failed
make: *** [ghcn.exe] Error 1
Tony,
Why are you using sprintf instead of ostringstream?
Jeff
sprintf provides a very simple and clean way to do precise formatting.
Just include stdio.h in the GHCN.h, worked for me on MinW32…
(no specialist here … :-) …)
when I execute the “get” file in “./get”, ubuntu (but also cygwin) responds by “invalid interpreter: file or map does not exist” … (I’m translating from Dutch here …)
what do I do wrong ?
thanks in advance …
Try this command: “bash ./get” (don’t type the quotes)
Also, if your distro doesn’t have “curl” you can use “wget” as a replacement, just remove the “-O” option when doing so.
Glenn, thank you my friend !
the “bash ./get” did the job and downloading started …
I have the ghcnd_hcn directory now, so I can go there …
has, among others, a “ghcnd-stations.txt” that I can open with gedit …
has “us.txt” … +/- 2.5 Gb ..
but the program “ghcn.exe” is only 11 bytes … would that be correct ? …
moreover, since it is an “exe”, WINE tries to open it … is that correct ?
when I try Tony’s 2 commands – to make US.csv, and to make the 0317.csv – TERMINAL says that “bash: ./ghcn.exe : file or map does not exist” …
isn’t the “ghcn.h” the program that takes care of everything ?
I’m close … I feel that … :-) … but not yet completely there …
or maybe I’m not as “close” as I thought … it will depend on “ghcn.exe” …
(“curl -0” seems to work though .. )
Size of 11 bytes might be correct for a link. Did a ‘make’ step produce any output?
Reporting everything on your screen after the ./get may be helpful to figuring things out. Also reporting whether you are on Ubuntu or under Cygwin when you do so would also help.
Neal, thanks already for your time … :-)
1. this is done on a UBUNTU 14.04 LTS pc … I’ve updated, and it is up to date, without any error messages …
2. this is what I get … the messages are in Dutch … and where it says, by executing the “ghcn.exe” program “bash: ./ghcn.exe: Bestand of map bestaat niet”, it translates to “file or folder does not exist”
here’s a copy of the terminal window … and I put in a few extra “dir”-commands to show what is there and what is not there …
(thanks in advance already !!!)
ubuntu12@ubuntu12:~$ cd Documenten
ubuntu12@ubuntu12:~/Documenten$ cd heller
ubuntu12@ubuntu12:~/Documenten/heller$ dir
ghcn_files ghcn.tar
ubuntu12@ubuntu12:~/Documenten/heller$ cd ghcn_files
ubuntu12@ubuntu12:~/Documenten/heller/ghcn_files$ dir
get ghcnd_hcn ghcnd-stations.txt GHCN.h Main.cpp
GHCN.cpp ghcnd_hcn.tar.gz ghcnd-version.txt GHCNMain.cpp Makefile
ubuntu12@ubuntu12:~/Documenten/heller/ghcn_files$ cd ghcnd_hcn
ubuntu12@ubuntu12:~/Documenten/heller/ghcn_files/ghcnd_hcn$ ./ghcn.exe US.txt > US.csv
bash: ./ghcn.exe: Bestand of map bestaat niet
ubuntu12@ubuntu12:~/Documenten/heller/ghcn_files/ghcnd_hcn$ dir
ghcnd-stations.txt USC00142401.dly USC00284229.dly USC00401790.dly
ghcn.exe USC00142459.dly USC00284987.dly USC00402024.dly
US_0317.csv USC00142835.dly USC00285728.dly USC00402108.dly
USC00011084.dly USC00143527.dly USC00286055.dly USC00402202.dly
the list, of course, goes on here … so, with the little I know about it, “ghcn.exe” exists in this folder, “US.txt” exists in this folder … it does make a “US.csv” – it’s size is 0 bytes – , but then … “ghcn.exe not found ” …
?
I had to make sure I had both curl and g++ installed
(That way I didn’t have to edit file to use wget)
apt install g++
apt install curl
I am on Kubuntu 16.??
This is what I got on my screen as I installed
—————-
maria@maria-ET1331G:~$ ls -l Downloads/ghcn.tar
-rw-rw-r– 1 maria maria 369152 Mar 21 23:22 Downloads/ghcn.tar
maria@maria-ET1331G:~$ tar -xvf Downloads/ghcn.tar
./._ghcn_files
ghcn_files/
ghcn_files/._get
ghcn_files/get
ghcn_files/._GHCN.cpp
ghcn_files/GHCN.cpp
ghcn_files/._GHCN.h
ghcn_files/GHCN.h
ghcn_files/._GHCNMain.cpp
ghcn_files/GHCNMain.cpp
ghcn_files/._Main.cpp
ghcn_files/Main.cpp
ghcn_files/._Makefile
ghcn_files/Makefile
maria@maria-ET1331G:~$ cd ghcn.tar
bash: cd: ghcn.tar: No such file or directory
maria@maria-ET1331G:~$ cd ghcn_files/
maria@maria-ET1331G:~/ghcn_files$ bash ./get
g++ -O3 -o ghcn.exe GHCNMain.cpp Main.cpp GHCN.cpp
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 278M 100 278M 0 0 2872k 0 0:01:39 0:01:39 –:–:– 2965k
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 8469k 100 8469k 0 0 2010k 0 0:00:04 0:00:04 –:–:– 2010k
ghcnd_hcn/
ghcnd_hcn/USC00011084.dly
ghcnd_hcn/USC00013160.dly
ghcnd_hcn/USC00013511.dly
ghcnd_hcn/USC00017157.dly
…..
ghcnd_hcn/USC00413873.dly
ghcnd_hcn/USC00415018.dly
ghcnd-version.txt
maria@maria-ET1331G:~/ghcn_files$ ./ghcn.exe US.txt date=0317 > US_0317.csv
Month = 3 Day = 17
maria@maria-ET1331G:~/ghcn_files$ ls -ltr
total 294144
-rwxr-xr-x 1 maria maria 200 Mar 17 08:37 Makefile
-rwxr-xr-x 1 maria maria 288 Mar 17 08:37 Main.cpp
-rwxr-xr-x 1 maria maria 297941 Mar 17 08:37 GHCNMain.cpp
-rwxr-xr-x 1 maria maria 46602 Mar 17 08:37 GHCN.h
-rwxr-xr-x 1 maria maria 9080 Mar 17 08:37 GHCN.cpp
-rwxr-xr-x 1 maria maria 262 Mar 17 08:37 get
-rw-r–r– 1 maria maria 270 Mar 22 09:47 ghcnd-version.txt
-rwxrwxr-x 1 maria maria 239604 Mar 22 23:08 ghcn.exe
-rw-rw-r– 1 maria maria 291861098 Mar 22 23:17 ghcnd_hcn.tar.gz
-rw-rw-r– 1 maria maria 8672498 Mar 22 23:18 ghcnd-stations.txt
drwxr-xr-x 2 maria maria 40960 Mar 22 23:20 ghcnd_hcn
-rw-rw-r– 1 maria maria 41 Mar 22 23:21 US_0317.csv
maria@maria-ET1331G:~/ghcn_files$
I flubbed at end … need to cd to ghcnd_hcn directory
——
maria@maria-ET1331G:~/ghcn_files$ cd ghcnd_hcn/
maria@maria-ET1331G:~/ghcn_files/ghcnd_hcn$ ./ghcn.exe US.txt date=0317 > US_0317.csv
Month = 3 Day = 17
United States
BREWTON 3 SSE AL USC00011084
FAIRHOPE 2 NE AL USC00012813
GAINESVILLE LOCK AL USC00013160
…..
well … haleluyah ! … :-)
1. Neal’s example didn’t do it … still the “ghcn.exe not known” …
2. thought it was Ubuntu 14 … since I had to do it anyway, upgraded to Ubuntu 16 …
3. everything was there … up to date … g++ … curl …
4. and I started anew … but all that didn’t have anything to do with the “real” solution (imho)
5. executing the first commands, up to the (bash)./get, I then got the same error as Jeff L. here above
6. so following the advice from Glenn … BEFORE you execute “…$ bash ./get” … edit “GHCN.h” to include “#include ” …
7. haven’t the faintest idea what it meant … (maybe visual studio ? …) but that did the trick …
8. there finally is a REAL “ghcn.exe” … now 250,8 kb in size …
thank you very much Jeff, Glenn and Neal !!!
ps … in point 6, you have to ignore Tony’s warning of not messing up his code
fok …
something got left behind in point 6 …
you have to include “#include ” in GHCN.exe
ok …
the open and close brackets do not come along …
just copy, in GHCN.exe, one of the lines “#include … “, and replace the existing with “stdio.h” between the brackets …
ok, the last one … :-)
in the previous 2 entries, “GHCN.exe” should be “GHCN.h”
!!!!
Cool, Tony! I will try this out!
Put it on GitHub.
I would be more than happy to help the development.
Tony,
Awesome. I’m waiting for comments from those who claim you don’t show your work; Zeke, Mosh, etc.
So that text doesn’t wrap so much …
History of commands
————–
119 tar -xvf Downloads/ghcn.tar
120 cd ghcn.tar
121 cd ghcn_files/
122 bash ./get
126 cd ghcnd_hcn/
127 ./ghcn.exe US.txt date=0317 > US_0317.csv
—–
And the tail end of above command
ASTORIA RGNL AP ,OR,USW00094224,1956,77,19.40,1963,76,-1.70
NEW YORK CNTRL PK TW,NY,USW00094728,1945,76,23.90,1916,77,-12.80
BLOCK ISLAND STATE A,RI,USW00094793,1973,76,12.80,1993,76,-11.10
UTICA ONEIDA CO AP ,NY,USW00094794,1990,76,18.90,1961,76,-17.20
PARK RAPIDS MUNI AP ,MN,USW00094967,2012,77,21.10,1939,76,-30.60
maria@maria-ET1331G:~/ghcn_files/ghcnd_hcn$ history | tail -20
————– and tail end of ls -ltr after above command
-rw-r–r– 1 maria maria 2612790 Mar 22 09:47 USW00094967.dly
-rw-r–r– 1 maria maria 3780810 Mar 22 09:47 USW00094728.dly
-rw-rw-r– 1 maria maria 2540878200 Mar 22 23:20 US.txt
lrwxrwxrwx 1 maria maria 11 Mar 22 23:20 ghcn.exe -> ../ghcn.exe
lrwxrwxrwx 1 maria maria 21 Mar 22 23:20 ghcnd-stations.txt -> ../ghcnd-stations.txt
-rw-rw-r– 1 maria maria 348 Mar 22 23:33 below_maximum_threshold_37.8_3170.kml
-rw-rw-r– 1 maria maria 1043 Mar 22 23:33 above_maximum_threshold_37.8_3170.kml
-rw-rw-r– 1 maria maria 270 Mar 22 23:33 below_minimum_threshold_-18_3170.kml
-rw-rw-r– 1 maria maria 270 Mar 22 23:33 received_snow_3170.kml
-rw-rw-r– 1 maria maria 224733 Mar 22 23:33 average_anomaly_trend_date_0317.kml
-rw-rw-r– 1 maria maria 224733 Mar 22 23:33 maximum_anomaly_trend_date_0317.kml
-rw-rw-r– 1 maria maria 228496 Mar 22 23:33 minimum_anomaly_trend_date_0317.kml
-rw-rw-r– 1 maria maria 680431 Mar 22 23:33 US_0317.csv
maria@maria-ET1331G:~/ghcn_files/ghcnd_hcn$
Hi Tony. I have spotted a bug in your software – nothing substantive, just a glitch in the labeling for the plots. How can I best communicate the fix to you. Or have you already posted it to GitHub or any other repository?