GHCN Software Release

I have released a new version of my GHCN software. It is much faster at obtaining NOAA data and should work on Mac, Linux or Cygwin on Windows.  Download the software here .

Then run these commands from a Linux, Mac or Cygwin command prompt.

tar xvf ghcn.tar
cd ghcn_files
chmod 755 get
./get
cd ghcnd_hcn
./ghcn.exe US.txt > US.csv

This is the software I wrote and use for analyzing the US temperature record. There are tons of command line options. For example, if you want to know what historical temperatures were on March 17 type :

./ghcn.exe US.txt date=0317 > US_0317.csv

As I mentioned, it has a huge range of possibilities for climate analysis, but you will have to dig through the C++ code to figure it out. I will give examples in my posts.

This entry was posted in Uncategorized. Bookmark the permalink.

22 Responses to GHCN Software Release

  1. Jimmy Haigh says:

    You lost me at “Cygwin”…

    • tonyheller says:

      Cygwin is a Gnu environment for Windows so you can use it like a UNIX system. Not recommended for non-UNIX users.

      Mac and Linux are UNIX based, so they have everything you need already installed.

  2. Steve Case says:

    Thanks Tony, but I’m not brave enough to try it.

    BUT if you were to put up a link the the your latest release of “Pulling Back the Curtain” that would be great. I think I didn’t download that last one. And the one I have won’t run anymore I assume because of all the automatic updates that we all get always seems to screw something up.

  3. Jeff L. says:

    $ make
    g++ -O3 -o ghcn.exe GHCNMain.cpp Main.cpp GHCN.cpp
    GHCN.cpp: In function ‘std::__cxx11::string dateString(int, int)’:
    GHCN.cpp:267:62: error: ‘sprintf’ was not declared in this scope
    sprintf(date, “%s %d”, MONTH_NAMES[month], days_remaining);
    ^
    Makefile:5: recipe for target ‘ghcn.exe’ failed
    make: *** [ghcn.exe] Error 1

    Tony,
    Why are you using sprintf instead of ostringstream?

    Jeff

    • tonyheller says:

      sprintf provides a very simple and clean way to do precise formatting.

    • Glenn says:

      Just include stdio.h in the GHCN.h, worked for me on MinW32…

      • ducdorleans says:

        (no specialist here … :-) …)

        when I execute the “get” file in “./get”, ubuntu (but also cygwin) responds by “invalid interpreter: file or map does not exist” … (I’m translating from Dutch here …)

        what do I do wrong ?

        thanks in advance …

        • Glenn says:

          Try this command: “bash ./get” (don’t type the quotes)
          Also, if your distro doesn’t have “curl” you can use “wget” as a replacement, just remove the “-O” option when doing so.

          • ducdorleans says:

            Glenn, thank you my friend !

            the “bash ./get” did the job and downloading started …

            I have the ghcnd_hcn directory now, so I can go there …

            has, among others, a “ghcnd-stations.txt” that I can open with gedit …
            has “us.txt” … +/- 2.5 Gb ..

            but the program “ghcn.exe” is only 11 bytes … would that be correct ? …
            moreover, since it is an “exe”, WINE tries to open it … is that correct ?

            when I try Tony’s 2 commands – to make US.csv, and to make the 0317.csv – TERMINAL says that “bash: ./ghcn.exe : file or map does not exist” …

            isn’t the “ghcn.h” the program that takes care of everything ?

            I’m close … I feel that … :-) … but not yet completely there …

          • ducdorleans says:

            or maybe I’m not as “close” as I thought … it will depend on “ghcn.exe” …

            (“curl -0” seems to work though .. )

          • neal s says:

            Size of 11 bytes might be correct for a link. Did a ‘make’ step produce any output?

            Reporting everything on your screen after the ./get may be helpful to figuring things out. Also reporting whether you are on Ubuntu or under Cygwin when you do so would also help.

          • duc dorleans says:

            Neal, thanks already for your time … :-)

            1. this is done on a UBUNTU 14.04 LTS pc … I’ve updated, and it is up to date, without any error messages …

            2. this is what I get … the messages are in Dutch … and where it says, by executing the “ghcn.exe” program “bash: ./ghcn.exe: Bestand of map bestaat niet”, it translates to “file or folder does not exist”

            here’s a copy of the terminal window … and I put in a few extra “dir”-commands to show what is there and what is not there …

            (thanks in advance already !!!)

            ubuntu12@ubuntu12:~$ cd Documenten
            ubuntu12@ubuntu12:~/Documenten$ cd heller
            ubuntu12@ubuntu12:~/Documenten/heller$ dir
            ghcn_files ghcn.tar

            ubuntu12@ubuntu12:~/Documenten/heller$ cd ghcn_files
            ubuntu12@ubuntu12:~/Documenten/heller/ghcn_files$ dir
            get ghcnd_hcn ghcnd-stations.txt GHCN.h Main.cpp
            GHCN.cpp ghcnd_hcn.tar.gz ghcnd-version.txt GHCNMain.cpp Makefile

            ubuntu12@ubuntu12:~/Documenten/heller/ghcn_files$ cd ghcnd_hcn
            ubuntu12@ubuntu12:~/Documenten/heller/ghcn_files/ghcnd_hcn$ ./ghcn.exe US.txt > US.csv
            bash: ./ghcn.exe: Bestand of map bestaat niet

            ubuntu12@ubuntu12:~/Documenten/heller/ghcn_files/ghcnd_hcn$ dir
            ghcnd-stations.txt USC00142401.dly USC00284229.dly USC00401790.dly
            ghcn.exe USC00142459.dly USC00284987.dly USC00402024.dly
            US_0317.csv USC00142835.dly USC00285728.dly USC00402108.dly
            USC00011084.dly USC00143527.dly USC00286055.dly USC00402202.dly

            the list, of course, goes on here … so, with the little I know about it, “ghcn.exe” exists in this folder, “US.txt” exists in this folder … it does make a “US.csv” – it’s size is 0 bytes – , but then … “ghcn.exe not found ” …

            ?

          • neal s says:

            I had to make sure I had both curl and g++ installed
            (That way I didn’t have to edit file to use wget)

            apt install g++
            apt install curl

            I am on Kubuntu 16.??
            This is what I got on my screen as I installed
            —————-
            maria@maria-ET1331G:~$ ls -l Downloads/ghcn.tar
            -rw-rw-r– 1 maria maria 369152 Mar 21 23:22 Downloads/ghcn.tar
            maria@maria-ET1331G:~$ tar -xvf Downloads/ghcn.tar
            ./._ghcn_files
            ghcn_files/
            ghcn_files/._get
            ghcn_files/get
            ghcn_files/._GHCN.cpp
            ghcn_files/GHCN.cpp
            ghcn_files/._GHCN.h
            ghcn_files/GHCN.h
            ghcn_files/._GHCNMain.cpp
            ghcn_files/GHCNMain.cpp
            ghcn_files/._Main.cpp
            ghcn_files/Main.cpp
            ghcn_files/._Makefile
            ghcn_files/Makefile
            maria@maria-ET1331G:~$ cd ghcn.tar
            bash: cd: ghcn.tar: No such file or directory
            maria@maria-ET1331G:~$ cd ghcn_files/
            maria@maria-ET1331G:~/ghcn_files$ bash ./get
            g++ -O3 -o ghcn.exe GHCNMain.cpp Main.cpp GHCN.cpp
            % Total % Received % Xferd Average Speed Time Time Time Current
            Dload Upload Total Spent Left Speed
            100 278M 100 278M 0 0 2872k 0 0:01:39 0:01:39 –:–:– 2965k
            % Total % Received % Xferd Average Speed Time Time Time Current
            Dload Upload Total Spent Left Speed
            100 8469k 100 8469k 0 0 2010k 0 0:00:04 0:00:04 –:–:– 2010k
            ghcnd_hcn/
            ghcnd_hcn/USC00011084.dly
            ghcnd_hcn/USC00013160.dly
            ghcnd_hcn/USC00013511.dly
            ghcnd_hcn/USC00017157.dly
            …..
            ghcnd_hcn/USC00413873.dly
            ghcnd_hcn/USC00415018.dly
            ghcnd-version.txt
            maria@maria-ET1331G:~/ghcn_files$ ./ghcn.exe US.txt date=0317 > US_0317.csv
            Month = 3 Day = 17
            maria@maria-ET1331G:~/ghcn_files$ ls -ltr
            total 294144
            -rwxr-xr-x 1 maria maria 200 Mar 17 08:37 Makefile
            -rwxr-xr-x 1 maria maria 288 Mar 17 08:37 Main.cpp
            -rwxr-xr-x 1 maria maria 297941 Mar 17 08:37 GHCNMain.cpp
            -rwxr-xr-x 1 maria maria 46602 Mar 17 08:37 GHCN.h
            -rwxr-xr-x 1 maria maria 9080 Mar 17 08:37 GHCN.cpp
            -rwxr-xr-x 1 maria maria 262 Mar 17 08:37 get
            -rw-r–r– 1 maria maria 270 Mar 22 09:47 ghcnd-version.txt
            -rwxrwxr-x 1 maria maria 239604 Mar 22 23:08 ghcn.exe
            -rw-rw-r– 1 maria maria 291861098 Mar 22 23:17 ghcnd_hcn.tar.gz
            -rw-rw-r– 1 maria maria 8672498 Mar 22 23:18 ghcnd-stations.txt
            drwxr-xr-x 2 maria maria 40960 Mar 22 23:20 ghcnd_hcn
            -rw-rw-r– 1 maria maria 41 Mar 22 23:21 US_0317.csv
            maria@maria-ET1331G:~/ghcn_files$

          • neal s says:

            I flubbed at end … need to cd to ghcnd_hcn directory
            ——
            maria@maria-ET1331G:~/ghcn_files$ cd ghcnd_hcn/
            maria@maria-ET1331G:~/ghcn_files/ghcnd_hcn$ ./ghcn.exe US.txt date=0317 > US_0317.csv
            Month = 3 Day = 17
            United States
            BREWTON 3 SSE AL USC00011084
            FAIRHOPE 2 NE AL USC00012813
            GAINESVILLE LOCK AL USC00013160
            …..

          • ducdorleans says:

            well … haleluyah ! … :-)

            1. Neal’s example didn’t do it … still the “ghcn.exe not known” …

            2. thought it was Ubuntu 14 … since I had to do it anyway, upgraded to Ubuntu 16 …

            3. everything was there … up to date … g++ … curl …

            4. and I started anew … but all that didn’t have anything to do with the “real” solution (imho)

            5. executing the first commands, up to the (bash)./get, I then got the same error as Jeff L. here above

            6. so following the advice from Glenn … BEFORE you execute “…$ bash ./get” … edit “GHCN.h” to include “#include ” …

            7. haven’t the faintest idea what it meant … (maybe visual studio ? …) but that did the trick …

            8. there finally is a REAL “ghcn.exe” … now 250,8 kb in size …

            thank you very much Jeff, Glenn and Neal !!!

            ps … in point 6, you have to ignore Tony’s warning of not messing up his code

          • ducdorleans says:

            fok …

            something got left behind in point 6 …

            you have to include “#include ” in GHCN.exe

          • ducdorleans says:

            ok …

            the open and close brackets do not come along …

            just copy, in GHCN.exe, one of the lines “#include … “, and replace the existing with “stdio.h” between the brackets …

          • ducdorleans says:

            ok, the last one … :-)

            in the previous 2 entries, “GHCN.exe” should be “GHCN.h”

            !!!!

  4. Steven Fraser says:

    Cool, Tony! I will try this out!

  5. Daniel Leite says:

    Put it on GitHub.
    I would be more than happy to help the development.

  6. gregole says:

    Tony,
    Awesome. I’m waiting for comments from those who claim you don’t show your work; Zeke, Mosh, etc.

  7. neal s says:

    So that text doesn’t wrap so much …
    History of commands
    ————–
    119 tar -xvf Downloads/ghcn.tar
    120 cd ghcn.tar
    121 cd ghcn_files/
    122 bash ./get

    126 cd ghcnd_hcn/
    127 ./ghcn.exe US.txt date=0317 > US_0317.csv

    —–
    And the tail end of above command
    ASTORIA RGNL AP ,OR,USW00094224,1956,77,19.40,1963,76,-1.70
    NEW YORK CNTRL PK TW,NY,USW00094728,1945,76,23.90,1916,77,-12.80
    BLOCK ISLAND STATE A,RI,USW00094793,1973,76,12.80,1993,76,-11.10
    UTICA ONEIDA CO AP ,NY,USW00094794,1990,76,18.90,1961,76,-17.20
    PARK RAPIDS MUNI AP ,MN,USW00094967,2012,77,21.10,1939,76,-30.60
    maria@maria-ET1331G:~/ghcn_files/ghcnd_hcn$ history | tail -20
    ————– and tail end of ls -ltr after above command
    -rw-r–r– 1 maria maria 2612790 Mar 22 09:47 USW00094967.dly
    -rw-r–r– 1 maria maria 3780810 Mar 22 09:47 USW00094728.dly
    -rw-rw-r– 1 maria maria 2540878200 Mar 22 23:20 US.txt
    lrwxrwxrwx 1 maria maria 11 Mar 22 23:20 ghcn.exe -> ../ghcn.exe
    lrwxrwxrwx 1 maria maria 21 Mar 22 23:20 ghcnd-stations.txt -> ../ghcnd-stations.txt
    -rw-rw-r– 1 maria maria 348 Mar 22 23:33 below_maximum_threshold_37.8_3170.kml
    -rw-rw-r– 1 maria maria 1043 Mar 22 23:33 above_maximum_threshold_37.8_3170.kml
    -rw-rw-r– 1 maria maria 270 Mar 22 23:33 below_minimum_threshold_-18_3170.kml
    -rw-rw-r– 1 maria maria 270 Mar 22 23:33 received_snow_3170.kml
    -rw-rw-r– 1 maria maria 224733 Mar 22 23:33 average_anomaly_trend_date_0317.kml
    -rw-rw-r– 1 maria maria 224733 Mar 22 23:33 maximum_anomaly_trend_date_0317.kml
    -rw-rw-r– 1 maria maria 228496 Mar 22 23:33 minimum_anomaly_trend_date_0317.kml
    -rw-rw-r– 1 maria maria 680431 Mar 22 23:33 US_0317.csv
    maria@maria-ET1331G:~/ghcn_files/ghcnd_hcn$

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.