From: Captain Jack on
"Keith (Southend)G" <keith_harris9(a)hotmail.com> wrote in message
news:fa3fe4ea-de63-4c94-a69f-f277d5a10fcd(a)o28g2000yqh.googlegroups.com...
On Jan 20, 7:43 pm, "Captain Jack" <CaptainJack1...(a)comcast.net>
wrote:
> "Keith (Southend)G" <keith_harr...(a)hotmail.com> wrote in message
The updated code is here:
>
> http://www.CaptainJack3s.com/Code/ogprog4.txt
>

> That link doesn't appear to work Jack.

Sorry, I'm typing with one eye closed (cataract surgery for me in a couple
of weeks, and I'll be as right as rain). That should have been 3d, not 3s,
as in:

http://www.CaptainJack3d.com/Code/ogprog4.txt

Man, my whole day has been like that, one little thing after another.
Probably should have stayed in bed... :-D

--
Jack


From: Keith (Southend)G on
On Jan 20, 8:03 pm, "Captain Jack" <CaptainJack1...(a)comcast.net>
wrote:
> "Keith (Southend)G" <keith_harr...(a)hotmail.com> wrote in message
>
> news:fa3fe4ea-de63-4c94-a69f-f277d5a10fcd(a)o28g2000yqh.googlegroups.com...
> On Jan 20, 7:43 pm, "Captain Jack" <CaptainJack1...(a)comcast.net>
> wrote:> "Keith (Southend)G" <keith_harr...(a)hotmail.com> wrote in message
>
> The updated code is here:
>
>
>
> >http://www.CaptainJack3s.com/Code/ogprog4.txt
>
> > That link doesn't appear to work Jack.
>
> Sorry, I'm typing with one eye closed (cataract surgery for me in a couple
> of weeks, and I'll be as right as rain). That should have been 3d, not 3s,
> as in:
>
> http://www.CaptainJack3d.com/Code/ogprog4.txt
>
> Man, my whole day has been like that, one little thing after another.
> Probably should have stayed in bed... :-D
>
> --
> Jack

No problem :-)

Glad to hear you enjoy this stuff.
Running the new version now, the text appears straight away when I
click the first button and the hour glass is showing with the both
buttons grayed out. Still 5 minutes+, the 2nd bit is almost instant.
Out of interest how much RAM do you have on the computer you're using?

Animation is very a very rewarding pastime, you must have lots of
patience. For many years I was on the committee of out local Video
Club, before that it was Cine (Super 8), I was one for making holiday
films (good for people who can't sleep!), but we had one member George
Daniels, who's been dead for some years, that made super animations. I
imagine it's all computer based now?

Keith (Southend)
From: Captain Jack on
"Keith (Southend)G" <keith_harris9(a)hotmail.com> wrote in message
news:4227f518-ad62-4993-8710-faf4a187618f(a)n7g2000yqb.googlegroups.com...
>
> Not sure if there's anything here that means more to you with regards
> to the time it takes to append the ogimet.txt to RichTextBox1. I tried
> an even larger file (18 hours worth), which took 11 minutes. The 2nd
> bit again was almost instant to run. Even so, I can just go and make
> myself a cup of tea and relax for a few minutes.

I don't think that would help with the problem we're having. That code
sample is for a quick way to build a really big string from other string
variables (and the StringBuilder is definitely the way to go for that). I'm
pretty sure the delay we're getting is happening somwhere when the
StreamReader is being created, which is a kettle of fish of a different
color.

>
> ps: I hope the Indianapolis Colts play well against the N Y Jets
> tomorrow (Sunday).

They did indeed do well, although they got off to a rocky start. They won
the game, which sends them to the Big Game in two weeks. There was much
cheering (and too much consumption of potato chips, I regret to say) in my
household. :-)

I set up a WebBrowser control to point to the data site on Saturday
afternoon, and I was able to load the page in a program. I need to do some
more work on it... I want to build the program so you can set up a list of
locations to pull data from, have it record the last date and time each
location was run for, and automatically download the data for each batch in
turn. The idea would be an application that would automatically get all of
the data you want since the last time you ran it, filter and sort the data
automatically, save its state, then shut down. My hope is that we'll have a
progam that you can run once a day (or as often as you like) and not have to
pay any attention to. From there, we could even set the program to go to
sleep every 24 hours, wake up and run, then go back to sleep until the next
day. If you start it when your computer boots up, you'd never have to mess
with collecting data again. At least, until they change the web site, which
some places have an annoying habit of doing.

I'll try to work on the project over the coming week and see what I can do.
If I get a chance, I'll try to isolate that start up issue with the inbound
file. If it is in the opening of the file, I'll see if there's another
reading option. If it's just slow on the reading because it's not buffering,
I can change to read in large blocks, and peel the lines off in memory (I've
done that with a lot of other unbuffered systems and it works well).

What version of VB are you using? I've got both 2005 and 2008 here. I'll
probably have more than one form before we're done, and it'll be easier if I
send you an entire solution zipped up, rather than just some code in text.

--
Jack



From: Captain Jack on
"Keith (Southend)G" <keith_harris9(a)hotmail.com> wrote in message
news:31c61827-d119-43f4-81ff-55a9dc235f47(a)a32g2000yqm.googlegroups.com...
>
> Sounds like you've given a lot of thought to the WebBrowser control
> for downloading that data. I'm quietly confident that the website I
> get the data from is unlikely to change it's address as I have had
> emails with the author in the past. In fact the address with
> "www1.ogimet...." he gave me as this presides on the University of
> Granada (Spain) website, the version without the '1' is on his home
> computer and regularily gets overloaded. So there are two variants of
> the same website:
> http://www1.ogimet.com/synopsc.phtml.en (best one to use)
> http://www.ogimet.com/synopsc.phtml.en
> I like the automatic running idea, certainly useful if I have to work
> away or go on holiday, always a nightmare catching up, I must admit I
> grab and use the data at different times of the day, although
> generally evenings. What if my internet went down, as it's on 24/7,
> would it just pick up from where it left? As long as I can set dates/
> times manually if I want to I don't see a problem. A 24 hour download
> file would be about 3.5Meg.
>
> I have installed 'Visual Basic 2008 Express', it came with a book I
> bought "Build a Program Now."
> http://www.microsoft.com/learning/en/us/book.aspx?ID=12282&locale=en-us
> Ironically, the main exercise is for a weather reporting programme :-)

What I was thinking of would be a table that would be kept in a text file on
your computer that would list each of the locations that get looked up, and
what the last date and time was for each one. When the program runs, it will
grab anthing after that date and time for each location up to the current
date and time, so that it would always be getting whatever hadn't been
retrieved before, regardless of how long it had been run. That would also
allow for adding new locations, or catching up if data was not available for
a certain location or if the download gets interrupted somehow.

There'll probably be a need to set up a couple of folders for working space
on the system, too, so the program has a safe place to store temporary
files, and a permanent place to store its logs. I'm thinking of making a
little window that will allow doing that kind of set up, although it would
be pretty easy to just put it into a text file and edit it by hand, too.

I think we could build in some monitoring for the WebBrowser that would
check the preferred web site, and only try the alternate site if the primary
site times out more than a specified number of times. Come to think of it,
we'll probably need a screen that shows the log of recent activity, so that
we'll know if no data is being grabbed.

--
Jack


From: Captain Jack on
"Keith (Southend)G" <keith_harris9(a)hotmail.com> wrote in message
news:f23b31c8-744d-46a7-9328-86c25795ab19(a)y12g2000yqh.googlegroups.com...
>
> One thing worth mentioning, the way the data on that site updates
> means that the full complement of data is normally 6 hours behind.
> eg, data up to say 14:00, won't all be in until 20:00, so I grab the
> data up to 14:00 and pick the rest the next day and so on. It updates
> every 6 hours as a rule, but you will find data there, but not all
> prior to this time, so I never actually go as far as the current time.
> That's probably as clear as mud :)
> It just dawned on me as I was thinking about the automation of the
> programme.

That shouldn't be a problem, we can just add an offset to the automation
that can be specified in the set up. To calculate the target time, the
program would just have to subtract the offset from the current time before
requesting the data.

Come to think of it, we might want a maximum time to download data, too. If
for some reason the program doesn't get run for a week, it might be better
to restrict a given download to a manageable size, like 24 or 48 hours. That
would keep the program from downloading too much data at one time, but still
let it catch up.

--
Jack