Apple Script Load Urls from .txt File and Get specified URLs

Question

Apple Script Load Urls from .txt File and Get specified URLs

Hello All

In the Automator exists a Action called "Get Specified URL's" which dies what i need, BUT every URL has to be entered separate.
I'd like to get the URL's from a .txt file or even better if possible from a excel file.

I read the text file with this:

set file_path to path to desktop folder as string
set file_path to file_path & "test.txt"
read file file_path
set file_contents to result

But how can i replicate the script from the library named "Get Specified URL's"?

Is there a way to show the real script behind the ones in the library? Would be good to learn from them.

Thanks everyone, every help highly appreciated.

iMac, macOS Sierra (10.12.4)

Posted on Mar 31, 2018 12:35 PM

Reply

Answer 1

Top-ranking reply

VikingOSX

Level 10

123,332 points

Mar 31, 2018 1:34 PM in response to Daniel Mueller1

The Automator actions are either compiled Objective-C code, or in some cases Python scripts.

The following AppleScript assumes a text file on the Desktop named urls.txt. Its contents are (one to a line):

http://www.apple.com

http://www.hp.com

http://www.oracle.com

The following AppleScript will read each line of the urls.txt file into its own list item, and then Safari will loop through this list opening each URL location in its own new tab, or window:

set urlFile to (path to desktop as text) & "urls.txt" as alias

-- place each URL entry in the input file as a list item

set urList to (read urlFile as «class utf8» using delimiter linefeed) as list

tell application "Safari"

activate

repeat with aURL in urList

open locationaURL

end repeat

end tell

return

You could place this within a a Run AppleScript action in Automator.

See this prior link on how to generate an AppleScript list from an Excel spreadsheet.

To better manage your system resources and available memory, you may want to restrict the number of URL that you open concurrently.

Reply

Answer 2

rccharles

Level 6

13,037 points

Apr 1, 2018 3:10 PM in response to rccharles

I think this does close to what you want. Let me know what is missing.

urls.txt is:

set urlFile to (path to desktop as text) & "urls.txt" as alias

-- place each URL entry in the input file as a list item
set urList to (read urlFile as «class utf8» using delimiter linefeed) as list

repeat with aURL in urList
    log "aURL is " & aURL
    sendMail(aURL)
end repeat

on sendMail(theURL)
   
    tell application "Mail"
       
        set theSubject to "Subject" -- the subject   
        set theContent to "Hallo hier die Links:" & theURL
        set theAddress to "sendToSomeone@no123.com" -- the receiver   
        set theSender to "legal@gtoclothing.com" -- your valid account email here   
        set msg to make new outgoing message with properties {subject:theSubject, content:theContent, visible:true, sender:theSender}
       
        tell msg to make new to recipient at end of every to recipient with properties {address:theAddress}
       
        send msg
       
    end tell
   
end sendMail

Reply

Answer 3

rccharles

Level 6

13,037 points

Mar 31, 2018 1:01 PM in response to Daniel Mueller1

Popular topic in the last month or two. There are several solutions here. You would link to the applescript if you needed to do something in automator.

Re: Can automator gather files listed in an excel spreadsheet?

How to create and run an applescript file.

Here is how to create an Applescript Application.

Example program. it's not a solution for this question. :

(*

https://discussions.apple.com/thread/3033221?tstart=30

Required:


Turn on accessability GUI in system preferences.

Optional:
--------
Full Key Codes
http://www.versiontracker.com/dyn/moreinfo/macosx/21215


youpi or iKey
   youpi and iKey are keystroke remapping programs.
   youpi, the free version, works fairly well for me in Mac OS 10.4 although not officially supported.
http://www.versiontracker.com/dyn/moreinfo/macosx/11485&vid=75326
   iKey is the paid version
http://www.scriptsoftware.com/ikey/

*)

on run
    -- Write a message into the event log.
    -- To see, run this applescript in Sript Editor. Click on Event Log tab at bottom of screen.
    -- Click run.
    log "  --- Starting on " & ((current date) as string) & " --- "

    set fromApp to "Safari"
    set toApp to "TextEdit"

    --- We have arrived.


    say "we are beginning now."

    -- Start your application
    tell application fromApp
        activate
    end tell

    delay 1

    say "copy data to clipboard"

    tell application "System Events"
        tell process fromApp


            keystroke "c" using command down
            delay 2

        end tell
    end tell


    -- Start your application
    tell application toApp
        activate
    end tell

    delay 5
    tell application "System Events"
        tell process fromApp
            --  open file menu
            keystroke return
            keystroke return
            keystroke return

            keystroke "v" using command down

        end tell
    end tell

    say "good news, We are done."

end run

The first thing that you need to do is to make the text into an AppleScript program.

Start the AppleScript Editor

/Applications/AppleScript/Script Editor.app

In Snow Leopard it's at: /Applications/Utilities/AppleScript Editor

Copy the script text to the Applescript editor.

Save the text to a file as an application and do not check any of the boxes below.

If you want access to the script from your Script Menu, move the script (the saved script application file) to your

~/Library/Scripts folder. You can also drag it to your Dock or make an alias for it on the Desktop.

To debug, run the script within the Applescript Editor. Click on the event log tab at the bottom of the window. Click on the run icon. The results from the log statement will be shown at the bottom of the screen.

R

Reply

Answer 4

rccharles

Level 6

13,037 points

Apr 4, 2018 11:43 AM in response to Daniel Mueller1

I do not understand step 4.

would it be possible to give a real working link so that I can try it out? I'd like to see what a link like "https://xyz.com/search/keyword" would return. The fact that it is a search link is a little worrisome to me. Want to try out the lynx command on a search.

Ok, here is your specification:

1. Read URLs from a Textfile like this:

https://xyz.com/search/keyword

https://xyz.com/search/keyword/page2

[understood. done.]

2. Fetch each URL in the Background one after another (no need to open them in the browser)

I do not have to open them in a browser if possible as i just want to get all the links from there

3. Get Link URL's from these pages

Get all the links these fetched pages contain

[ lynx fetches the page and pulls out the links. so I get steps 2 & 3 in one command. I this what you expect for a list of links. I trimmed the results. But this should convey the idea. ]

mac $ lynx -listonly -nonumbers -dump https://www.rcn.com/dc-metro/bundles/
   Visible links
https://www.googletagmanager.com/ns.html?id=GTM-PG7CDJ
tel:800-746-4726
https://www.rcn.com/dc-metro/bundles/
https://www.rcn.com/business
https://my.rcn.com/login
https://www.rcn.com/hub/customer-center/webmail/
https://www.rcn.com/hub/help/
https://www.rcn.com/hub/customer-center/contact-us/
https://www.rcn.com/hub/about-rcn/
https://www.rcn.com/rcntv/
https://www.rcn.com/
https://www.rcn.com/dc-metro/
https://www.rcn.com/boston/bundles/
https://www.rcn.com/chicago/bundles/
https://www.rcn.com/dc-metro/bundles/
https://www.rcn.com/lehigh-valley/bundles/
https://www.rcn.com/new-york/bundles/
https://www.rcn.com/philadelphia/bundles/
https://www.rcn.com/dc-metro/bundles/
https://www.rcn.com/dc-metro/special-offers/
https://www.rcn.com/dc-metro/special-offers/
https://www.rcn.com/dc-metro/high-speed-internet/
https://www.rcn.com/dc-metro/high-speed-internet/#shop
https://www.rcn.com/dc-metro/high-speed-internet/#equipment
https://www.rcn.com/dc-metro/high-speed-internet/#networking-security

   Hidden links:
https://shopnow.rcn.com/
https://www.rcn.com/dc-metro/bundles/
https://www.rcn.com/dc-metro/bundles/
https://www.rcn.com/dc-metro/bundles/#mobile-nav
https://twitter.com/rcnconnects
https://www.facebook.com/RCN
https://plus.google.com/101138942055455568681/about
https://www.youtube.com/user/RCNconnects
http://instagram.com/rcnconnects
mac $

don't really know what hidden links are.

4. Filter URL's

Filter the URL's for a word containing. For example "Voodoocats" and "Article"

I'm not sure about this step. Are you saying the link will contain the words "Voodoocats" and "Article"? Or, perhaps the page the link is pointing too?

R

Reply

Answer 5

Mar 31, 2018 1:22 PM in response to VikingOSX

Thank you VikingOSX

Getting me closer, maybe i should explain better what i'm trying to do.

I have it working with Atomator Actions, but it's a lot of work to enter the "specified URLs" as you can't paste more then one at a time and no way to import from a txt file or so.

Read URLs from a Textfile
Fetch each URL in the Background one after another (no need to open them in the browser)
Get Link URLs from these pages
Filter URLs
Write them into a Textfile or save as variable
Create a Email in Mail App and paste the filtered URLs into the message body
Would be handy to create the Mail Body like : Hello These are the links from Website A "var from textfile A.txt"

These functions are more or less available as Automator actions, but as i can't customize them i am a bit lost.

Thank you for your kind Help

Reply

Answer 6

Apr 2, 2018 9:04 AM in response to rccharles

Hello Charles

thank you again for your kind help. Why i am still using Automator is, that i have more functions there which i need and do not know how to execute them with a script.

I try to explain again.

1. Read URLs from a Textfile like this:

https://xyz.com/search/keyword

https://xyz.com/search/keyword/page2

2. Fetch each URL in the Background one after another (no need to open them in the browser)

I do not have to open them in a browser if possible as i just want to get all the links from there

3. Get Link URL's from these pages

Get all the links these fetched pages contain

4. Filter URL's

Filter the URL's for a word containing. For example "Voodoocats" and "Article"

5. Save this filtered URL's as variable "VoodoocatsLinks" for later use

I need these links later on to paste them into the emails body/message

6. Repeat steps 1-5 with different url's in a different text file and filter for different keywords

Repeat these steps and save the resulting url's as variable "dog" and use them later in the Email msg.

7. Create a mail containing a body/message like:

"Dear mr. XY

I'm sending you the links as following: Voodocats: (All links from variable "VoodoocatsLinks")

and here are the links for Dog: (All links from variable "DogLinks")"

8. Send Email

BTW, i'm more then happy to pay someone for this, but i have no idea where to hire a Apple Script dev.? Any tips for this?

Have a good day Charles and thanks again for your great help with this

Daniel

Reply

Answer 7

rccharles

Level 6

13,037 points

Apr 2, 2018 1:19 PM in response to Daniel Mueller1

Your explanation of the requirements is much more detailed.

Which of the steps have you gotten to work?

I'm busy the next two days, but will look at it in more detail after that.

This site might have a tool for extracting the links.

Online Tool to Extract Links from any Web Page | HackerTarget.com

R

Reply

Answer 8

rccharles

Level 6

13,037 points

Apr 4, 2018 11:53 AM in response to rccharles

An example would clarify things.

5. Save this filtered URL's as variable "VoodoocatsLinks" for later use

I need these links later on to paste them into the emails body/message

[ any reason why we cannot send the email now? Why save them in variable for later. This complicated the code.

6. Repeat steps 1-5 with different url's in a different text file and filter for different keywords

[Where do I get the words to filter? ]

Repeat these steps and save the resulting url's as variable "dog" and use them later in the Email msg.

[ are you saying to combine found url's from more than one page. How do I determine which pages to combine ? ]

7. Create a mail containing a body/message like:

"Dear mr. XY

R

Reply

Answer 9

rccharles

Level 6

13,037 points

Mar 31, 2018 2:36 PM in response to Daniel Mueller1

the email.

Intro to applescript with sending an email

https://gist.github.com/Moligaloo/3850710

Reply

Answer 10

Mar 31, 2018 3:22 PM in response to rccharles

Hello rccharles

Thank you for the links. i am using some of the infos from this gist already.

For the sake of me, i can't paste a variable i have stored before into the email body with:

tell application "Mail"

set VoodoocatsLinks to (get value of variable "Voodoocats" of front workflow)

set theSubject to "Subject" -- the subject

set theContent to "Hello here are the links: VoodoocatsLinks" -- the content

set theAddress to "xxx@163.com" -- the receiver

set theSignatureName to "signature_name"-- the signature name

set theAttachmentFile to "Macintosh HD:Users:moligaloo:Downloads:attachment.pdf" -- the attachment path

set msg to make new outgoing message with properties {subject: theSubject, content: theContent, visible:true}

tell msg to make new to recipient at end of every to recipient with properties {address:theAddress}

tell msg to make new attachment with properties {file name:theAttachmentFile as alias}

set message signature of msg to signature theSignatureName

send msg

end tell

Reply

Answer 11

rccharles

Level 6

13,037 points

Apr 1, 2018 3:28 AM in response to Daniel Mueller1

-- notice the change.

set theContent to "Hello here are the links: " & VoodoocatsLinks

Reply

Answer 12

Apr 1, 2018 3:33 AM in response to rccharles

Hello rccharles

Thanks, i completely missed this! Unfortunately it' still giving me headaches and i tried 8hours to fix it with infos in threads i have found. But my variables are not returned and it returns only:

Hallo Hier die Links: current applicationcurrent applicationcurrent applicationcurrent applicationcurrent applicationcurrent applicationcurrent....

I checked the variables with a "show result" after i have saved them and they do look good. They contain about 10 Url's.

Heres my updated script:

set VoodoocatsLinks to get value of variable "Voodoocats" of front workflow

tell application "Mail"

set theSubject to "Subject" -- the subject

set theContent to "Hallo hier die Links:" & VoodoocatsLinks-- the content

set theAddress to "xxx@163.com" -- the receiver

set theSender to "legal@gtoclothing.com" -- your valid account email here

set msg to makenewoutgoing messagewith properties {subject:theSubject, content:theContent, visible:true, sender:theSender}

-- select signature for Mail

tell application "Mail" to activate

tell application "System Events"

tell process "Mail"

clickpop up button 2 of window 1

delay 0.01

keystroke "G"

delay 0.01

keystrokereturn

delay 0.01

end tell

tell msg to makenewto recipientat end of every to recipientwith properties {address:theAddress}

--send msg

end tell

And here the complete Workflow in Automator:

Reply

Answer 13

rccharles

Level 6

13,037 points

Apr 1, 2018 2:48 PM in response to Daniel Mueller1

Why are you continuing to use automator?

Don't you have all the applescript code that you need?

R

Reply