Way to automate web form entry & data retreival?

I'd like to create an action (?) that, every night, collects my 401(k) data and inserts it into a spreadsheet. I'm not sure Automator is the answer, but maybe someone else has a better option. Here's what needs to be done:

1. Go to the main web page and click the "Log On" button
2. Type SS# and password into fields in a form [tricky part - must wait for web page to fully load] then click another "log on" button
3. Summary page now up. Click on link to "fund performance" section.
4. Once that page loads, copy key text ($$ values) and eventually paste it into a spreadsheet.

I tried this with a few macro programs and even Applescript, but I had trouble waiting for the pages to fully load before typing. I also couldn't get a smooth flow from copying the data to pasting it into Excel.

Does anyone have any advice on how to go about doing this, or maybe a pointer to another source of help?

Thank you,

Jeff

Dual 1GHz G4 MDD, Mac OS X (10.4.3)

Posted on Mar 1, 2006 8:53 PM

Reply
4 replies

Apr 21, 2006 8:47 PM in response to Will Jackson

Hi - sorry I'm responding to this a month after you wrote it, but I just discovered my old thread had new posts!

OK, I don't know how easy this will be, but I'm going to paste in my Applescript code and edit out my personal information. This is my personal financial information after all, so I'm going to be a bit conservative on the editing:

tell application "Safari"
activate
make new document
set URL of document 1 to "http://### My 401k website ###"
end tell

delay 10

-- This part simply hits an "Enter" button on the welcome screen

tell application "System Events"
tell application process "Safari"
keystroke return -- hit the enter key via default web form behavior
end tell
end tell


delay 10

-- Here is the login page. Username/password, then hit enter:

tell application "System Events"
tell application process "Safari"
keystroke "### My user ID ##"
keystroke tab
keystroke "### My password###"
keystroke return -- hit the enter key via default web form behavior
end tell
end tell


delay 10

-- OK, here we're at the "overview" page of my 401k website. I extract some pertinent information like balance, %gain, etc. and put them into variables. I used a GUI scripting tool (not Apple's) to get the path to the piece of text I was looking for:

tell application "System Events"
tell application process "Safari"

set contrib to item 1 of (get name of (static text of group 57 of UI element 1 of scroll area 1 of group 4 of window "Your Benefits Resources - 401(k) Savings"))
set gains to item 1 of (get name of (static text of group 59 of UI element 1 of scroll area 1 of group 4 of window "Your Benefits Resources - 401(k) Savings"))
set bal to item 1 of (get name of (static text of group 64 of UI element 1 of scroll area 1 of group 4 of window "Your Benefits Resources - 401(k) Savings"))
set rate_return to item 1 of (get name of (static text of group 62 of UI element 1 of scroll area 1 of group 4 of window "Your Benefits Resources - 401(k) Savings"))

-- Here I click a link to take me to the detail page of my 401k account:

click static text "Your Investment and Account Summaries" of UI element "Your Investment and Account Summaries" of group 66 of UI element 1 of scroll area 1 of group 4 of window "Your Benefits Resources - 401(k) Savings"
end tell
end tell


delay 10

-- And here I extract all the data - date, balances, # of shares, etc. and put them into variables:

tell application "System Events"
tell application process "Safari"

set closing_date to text 23 thru 32 of (item 1 of (get name of (static text of group 62 of UI element 1 of scroll area 1 of group 4 of window "Investment and Account Summaries - Investment Summary")))

set part1 to closing_date & tab & contrib & tab & gains & tab & bal & tab & rate_return & tab

set intbond_sh to item 1 of (get name of (static text of group 90 of UI element 1 of scroll area 1 of group 4 of window "Investment and Account Summaries - Investment Summary"))
set balanced_sh to item 1 of (get name of (static text of group 101 of UI element 1 of scroll area 1 of group 4 of window "Investment and Account Summaries - Investment Summary"))
set inteqty_sh to item 1 of (get name of (static text of group 112 of UI element 1 of scroll area 1 of group 4 of window "Investment and Account Summaries - Investment Summary"))
set smallcap_sh to item 1 of (get name of (static text of group 123 of UI element 1 of scroll area 1 of group 4 of window "Investment and Account Summaries - Investment Summary"))
set russell_sh to item 1 of (get name of (static text of group 134 of UI element 1 of scroll area 1 of group 4 of window "Investment and Account Summaries - Investment Summary"))
set jjstock_sh to item 1 of (get name of (static text of group 152 of UI element 1 of scroll area 1 of group 4 of window "Investment and Account Summaries - Investment Summary"))
set jjcontrib_sh to item 1 of (get name of (static text of group 170 of UI element 1 of scroll area 1 of group 4 of window "Investment and Account Summaries - Investment Summary"))

set intbond_pr to item 1 of (get name of (static text of group 91 of UI element 1 of scroll area 1 of group 4 of window "Investment and Account Summaries - Investment Summary"))
set balanced_pr to item 1 of (get name of (static text of group 102 of UI element 1 of scroll area 1 of group 4 of window "Investment and Account Summaries - Investment Summary"))
set inteqty_pr to item 1 of (get name of (static text of group 113 of UI element 1 of scroll area 1 of group 4 of window "Investment and Account Summaries - Investment Summary"))
set smallcap_pr to item 1 of (get name of (static text of group 124 of UI element 1 of scroll area 1 of group 4 of window "Investment and Account Summaries - Investment Summary"))
set russell_pr to item 1 of (get name of (static text of group 135 of UI element 1 of scroll area 1 of group 4 of window "Investment and Account Summaries - Investment Summary"))
set jjstock_pr to item 1 of (get name of (static text of group 153 of UI element 1 of scroll area 1 of group 4 of window "Investment and Account Summaries - Investment Summary"))
set jjcontrib_pr to item 1 of (get name of (static text of group 171 of UI element 1 of scroll area 1 of group 4 of window "Investment and Account Summaries - Investment Summary"))

set part2 to intbond_sh & tab & intbond_pr & tab & balanced_sh & tab & balanced_pr & tab & inteqty_sh & tab & inteqty_pr & tab & smallcap_sh & tab & smallcap_pr & tab & russell_sh & tab & russell_pr & tab & jjstock_sh & tab & jjstock_pr & tab & jjcontrib_sh & tab & jjcontrib_pr & return

-- This and the previous statements concatenated all my variables into one string to be appended to my text file:

set alldata to part1 & part2


end tell
end tell

-- Now I close the Safari window and get to writing out my text file:

tell application "System Events" -- Exit the system
tell application process "Safari"
click image 1 of UI element 1 of group 48 of UI element 1 of scroll area 1 of group 4 of window "Investment and Account Summaries - Investment Summary"
delay 6
keystroke "w" using command down
end tell
end tell

-- Open the appendable file and write the current day's data
-----------------------------------------------------------------------


set thefile to "### My path... ###:Documents:Excel:401k.txt"

try
close access file thefile
end try

open for access file thefile with write permission

write alldata to file thefile starting at eof
close access file thefile



I hope this helps. The biggest learnings for me were the technique for appending to a file and the method for accessing the UI web page fields.

Feel free to ask any more questions, but I'm as dumb as a stump when it comes to Applescripting. What I did here was just copying and learning from other sources. Good luck!

-Jeff


Dual 1GHz G4 MDD Mac OS X (10.4.3)

Dual 1GHz G4 MDD Mac OS X (10.4.3)

This thread has been closed by the system or the community team. You may vote for any posts you find helpful, or search the Community for additional answers.

Way to automate web form entry & data retreival?

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple Account.