I have made quite a few updates to my GHCN code, which is primarily targeted at the US HCN database – though it will work with any GHCN data set. The release includes seven files, compressed into file ghcn.tgz
getdaily GHCN.cpp GHCN.h ghcnd-stations.txt Main.cpp Makefile stations.txt
getdaily is a tcsh script for downloading and organizing the latest HCN daily temperature database from NOAA. It takes about an hour to run over a very fast link. It generates 49 .txt files – one for the US and one for each of Obama’s 57 states.
The code is C++ and consists of three files : Main.cpp GHCN.cpp GHCN.h. It should compile without warnings on g++ or Visual Studio compilers. The Makefile works on any gnu system, like Linux. On Cygwin, the suffix “.exe” gets added. To use Visual Studio, you will have to create a project and import the three code files.
There are a lot of new options
stations.txt must be in the same directory as the executable. Typical usage would be
./ghcn.exe NY.txt > NY.csv
That reads all NY records and outputs the results to a NY.csv file
The rest comes in the category of “either you can figure it out now, or you need two years of education to figure it out”
My critics will enjoy looking through this and seeing that (unlike them) I am not doing anything to bias the output data one way or another. That is because (unlike them) I am an actual scientist – and an honest person.
Thanks!
I’ll bite.
Obama’s 57 states…?
I hope Canada isn’t one of them.
Major blunder during campaign speech. Obama didn’t even know how many states the US has. There are 57 Islamic states, which makes the blunder even worse.
But the real quote was to have visited 57 states with one to go, as I remember.
Actually, 2. Hawaii and Alaska. So he thought there were 59.
I have my doubts that it was a blunder.
And I thank you for all you do.
It does not matter how right you are, if your are a heretic. Like the witches of bygone, either you drown or float when thrown in the pond.
The Climate Liars will never reveal their coding, data or anything else. They are sociopathic scum pretending to do science in order to inflict their sick, twisted political agenda on the world. These people are very dangerous. They are lunatics.
I can read everything fine in Visual Studio 2008, and it says build successful, but I don’t see what happens after that. I have the 2 .cpp and 1 .h visible, and I tell it to build it. Any tips on how I can turn your code into an executable, like a list of commands I need to do from VS? Could you go ahead and make the .exe and post it? (I would prefer to build myself in case I get smart enough to edit your code).
I have a linux laptop I could use after that (it won’t run Windows anymore, hardware problem).
I don’t know C++ (obviously), but a nice visual interface for this would be fun to work on. If it was easy enough, the world would be doing a whole lot more auditing and asking some pretty difficult questions.
Is there a way to make your program work from a DOS prompt, even just to pull in the daily data (run everything on a windows machine?)
I tried doing something like the getdaily in excel, and it would bomb after a few hours, but would successfully read and output the text files for many stations before it died (I was only using it as a VBA interface, not storing any data. I had to do some text translation too so Windows could work with it). It was totally unpredictable how many hours it would run before excel would crash. Longest I got was about 7 hours and it was maybe 1/3 of the way done. So it was really slow too, even on a 50 mbs connection..
I’d really like to get this to work if you can give a few steps to a C++ rookie. It might help others too if you had a step by step. This is very important work so the more participants the better.
On Linux, all you have to do is cd into the directory and type “make”
The executable will be named “ghcn”
You can run that from from the bash prompt. For Texas it would be
Download the files
./getdaily
Run the executable
./ghcn TX.txt > Tx.csv
Thanks I’ll fire up my old machine and try it.
The Makefile contains the magic, I guess. Just manage to find a system where:
> c++ -o ghcn Main.cpp GHCN.cpp
makes sense.
On FreeBSD I simply run
> clang++ -O3 -march=k8-sse3 -fomit-frame-pointer -ffast-math -fno-strict-aliasing -o ghcn_clang Main.cpp GHCN.cpp
g
-g?
ls -l ./ghcn*
-rwxr-xr-x 1 bark fart 179208 Apr 14 22:51 ghcn_clang3.5.bin
-rwxr-xr-x 1 cat meat 1421832 Apr 14 23:03 ghcn_clang3.5.bin-g
That’s an order of magnitude to handle a bunch of unusable debugging symbols. I dunno about the speed, but I’m sure it’s not _faster_.
g++ not c++
g++ is just the GNU c++ compiler. clang++ works (I just tried it) & “c++” is the canonical name for whatever c++ compiler you happen to have on your system (& which here is linked to clang++, not g++):
QED 😉
The makefile says g++, not c++
Your original comment made no sense
the -o argument is the name of the output file (so, if you need spaces in the name, you would put “-o Harry\ Harryhausen\ something” because the blackslash allows a space to go in the filename.
Parenthetically, spaces are delimiters, so don’t use them.
So, here’s what it is:
c++ is the compiler
-o whatever is the file that it creates
-f & -m & -O arguments are compiler specific & create a faster, or smaller executable.
& all the stuff at the end is the .cpp files you are trying to compile.
There are easier ways to do it, but it works.
Thanks, just got my ubuntu machine all updated (which took forever)… Hopefully it will run it.
Epic. The script kiddies of social media marketing have *only* mere marketing under their belt and lack all substance. But they are mere men made into girls by college LIBERAL Arts requirements and mandatory visits to the college Psyco-The-Rapist (psychotherapist) if they ever own their own balls. It used to be, guys balanced out girls, nation-wide. Then Society actually offered not only non-land owners the democratic (with a big D always) vote, but also hormonally insane *girls*! They immediately instituted Prohibition which by ironic effect created gangsterism and 3000 speakeasys in Washington, D.C. alone, and politics has never been again the same game of socialist workers party Democratic KKK party members against simple hands-off libertarians who enjoyed also the vice of hellfire Religion. Now they, on the Progresive Left, and the retrograde Right want to put the Internet viral Tea Party back in the box. But as late R.I.P. climate skeptical progressive lefty icon Alexander “Hot Cock” Cockburn exasperated in a video interview: “they have all the guns.”
Donald Trump 2016.
-=NYC=-
P.S. Judith Curry isn’t a passive aggressive bitch anymore, she’s Joan of Arc. And Lord Mockton isn’t a mere puzzle game bumbling old folk’s home retrograde emotional throwback, but a competent real world scientist. These recent changes are also epic.
That doesn’t look like C++. What language was your post written in?
I think you need to take DOM or Mescaline to decipher it but I am too busy right now to experiment with that hypothesis
It’s called English, bad ass flyboy.
Is there a translation guide somewhere for the language, “English bad ass flyboy”
On second thought I have no need to translate it
Watch it Nik, not all of us girls vote for the idiotic progressives. Think about ‘dave’ and ‘steve’ and the cross dresser ‘Snow White’. they aren’t girls but I am sure they are Progressives.
There is the Internet, the *only* culture left for dispersed populated rural West or East countries. Then there is something else entirely. There is real world high population density cities where different *types* of people get along just fine, no cultural turning into real culture war required. The cities represent peace in our time. Steve *pretends* that laypeople are NAZIs, as *if* the type of socially disconnected pressures of 1930s Germany exist now top in our day, laughably. But…Culture War. Yeah, whatever. Normal people have taken over Szep-i-Zizm, Iron Man and Ancient Gods idiot trivial fractal pattern matcher, and Steve Godwin’s desperate desire to fight a good war against real NAZIs comes out as total fodder and PR disaster idiocy. EVEN TONY WATTS DIRECTLY DISSES HIS MAIN THESIS AS ODD CHERRY PICKING. I didn’t do that. Tony did that.
Armies always consist of people who want to end the war, and others who want to keep it going.
Thanks for thinking.
Steve,
The people who ARE CANNON FODDER or whose sons are cannon fodder want to end the war.
The Sociopaths sitting behind the lines making money hand over fist from the war want the war to continue.
Notice how the USA has been almost continuously in wars since the Federal Reserve Act of 1913?
Steven, I altered your code to add .5 degrees every 5 years since 1979 so your data could be accurate… for data prior to 1979 I deducted .3 degrees per decade going back to 1910 ..
Finally your data matches the models.
Oh dear, now you’ve done it Philippe,
Didn’t anyone tell you, DON’T FIX THE MODELS!!!
If you fix the models, they will look just like the observations, and the whole Climate Industrial Complex will come crashing down. $21,000,000,000 worth of alarmists out of work. Harrison Ford and James Cameron, out of work. Bill McKibben, out of work. Joe Romm, out of work. Windmills spinning to the ground. Coal plants sprouting up all over.
Where will it all end?
Thanks Steve,
It will give me something to think about as the thaw (hopefully) sets in.
& yeah, it compiles pretty cleanly:
Stallman.
I will have to give that to my husband. And yes we know Stallman.
Yeah, I didn’t access Flickr now because Firefox did allowed my cookies this time.
Steve, here is what I’m getting using ubuntu and tcsh (keep in mind I have spent 10 minutes using unix type stuff in the last 20 years):
Reading state information… Done
The following NEW packages will be installed:
tcsh
0 upgraded, 1 newly installed, 0 to remove and 4 not upgraded.
Need to get 521 kB of archives.
After this operation, 1,402 kB of additional disk space will be used.
Get:1 http://us.archive.ubuntu.com/ubuntu/ precise/universe tcsh i386 6.17.06-2 [521 kB]
Fetched 521 kB in 0s (693 kB/s)
Selecting previously unselected package tcsh.
(Reading database … 159907 files and directories currently installed.)
Unpacking tcsh (from …/tcsh_6.17.06-2_i386.deb) …
Processing triggers for man-db …
Setting up tcsh (6.17.06-2) …
update-alternatives: using /bin/tcsh to provide /bin/csh (csh) in auto mode.
michael@michael-Inspiron-6000:~/Documents/AGW/release$ tcsh
michael-Inspiron-6000:~/Documents/AGW/release> getdaily
getdaily: Command not found.
michael-Inspiron-6000:~/Documents/AGW/release> exit
exit
michael@michael-Inspiron-6000:~/Documents/AGW/release$ tcsh
michael-Inspiron-6000:~/Documents/AGW/release> ./getdaily
./getdaily: Command not found.
michael-Inspiron-6000:~/Documents/AGW/release>
More:
michael@michael-Inspiron-6000:~/Documents/AGW/release$ ./getdaily
bash: ./getdaily: /usr/bin/tcsh^M: bad interpreter: No such file or directory
michael@michael-Inspiron-6000:~/Documents/AGW/release$ getdaily
getdaily: command not found
michael@michael-Inspiron-6000:~/Documents/AGW/release$ ./ getdaily
bash: ./: Is a directory
michael@michael-Inspiron-6000:~/Documents/AGW/release$ ./getdaily
bash: ./getdaily: /usr/bin/tcsh^M: bad interpreter: No such file or directory
michael@michael-Inspiron-6000:~/Documents/AGW/release$ dir
getdaily ghcnd-stations.txt Main.cpp stations.txt
GHCN.cpp GHCN.h Makefile
michael@michael-Inspiron-6000:~/Documents/AGW/release$
Any suggestions? I thought the problem was in /usr/bin, but tcsh is there now that I installed it. Is there a way I can invite you to take over my machine?
Type “which tcsh” (without the quotes) to find out where it is located on your computer, and replace the path to tcsh in file getdaily, with the one on your computer
It says \usr\bin\tcsh which is where ‘getdaily’ wants it to be, and I can see tsch there. I wonder if the problem is the next line get_daily.tcsh . Where is that file?
That line should have a # comment at the front of it. Just delete the line.
# get_daily.tcsh
Got a little further. From bash:
michael@michael-Inspiron-6000:~/Documents/AGW/release$ /usr/bin/tcsh getdaily
: Command not found.
: Command not found.
rm: No match.
: Command not found.
foreach: Words not parenthesized.
michael@michael-Inspiron-6000:~/Documents/AGW/release$
from tsch:
michael-Inspiron-6000:~/Documents/AGW/release> /usr/bin/tcsh getdaily
: Command not found.
: Command not found.
rm: No match.
: Command not found.
foreach: Words not parenthesized.
michael-Inspiron-6000:~/Documents/AGW/release>
Sorry if I’m being too dense> 🙂
Try running dos2unix on the file. It may have windows end of line characters
> dos2unix getdaily
HA! It is running now. Thanks Steve. Might have more questions after getdaily runs. Thanks so much.
It ran and ended, but it only took maybe an hour. I have a total of 1196 files, but there are 22206 stations in stations.txt. So I should have 22206 .dly files + original + 49 state & USA files, correct? I do have the state files, so maybe the server just doesn’t have anything for some of the .dly files? Looks like the state.txt files are good. Why so few files?
Should be 1,140 stations in stations.txt
Ah, yes, I was reading ghcnd-stations.txt. So the stations.txt only contains 1140 (of the 22208 in ghcnd), probably only the ones with full records going back a long time? What selection criteria did you use to make stations.txt?
Those are the HCN stations used by USHCN and NCDC
So getdaily is done, now I get:
michael@michael-Inspiron-6000:~/Documents/AGW/release$ make
g++ -O3 -o ghcn.exe Main.cpp GHCN.cpp
make: g++: Command not found
make: *** [ghcn.exe] Error 127
michael@michael-Inspiron-6000:~/Documents/AGW/release$
Thanks
sudo apt-get install build-essential g++
OK, now I have a NY.csv file! Lots of numbers, I’ll have to look at the other code and see what it means, then I’ll try to make some charts. I’m rebuilding an evil CO2 producing Land Rover V8 while I do this. Thanks for your help today.
OK THANKS!