Month: September 2016

A script to get a list of links off a given page. I found these two examples to start from: http://normansolutions.co.uk/post/using-powershell-to-check-all-pages-in-website, https://www.petri.com/testing-uris-urls-powershell.

clear
function get-URLlist($testPage, [int]$linkDepth){
    #set the counter
    [int]$i = 0     #loop thru the list of links, going to sub-pages until the counter equals the given link depth
    while($linkDepth -gt $i) {
        foreach($link in $testPage.Links) {
            #only look at links on the same site
            if($link.href.ToString() -notlike "http*") {
                #if not already in the array, add it
                if(!($sitemap.ContainsKey($link.innerText))){
                    $sitemap.add($link.innerText, $url + "/" + $link.href)
                    $testPage = Invoke-WebRequest "http://$url"
                    #check out the sub-pages one less level than the given link depth
                    $sitemap = get-URLlist -testPage $testPage -linkDepth $($linkDepth-1)
            }
        }
    }
    $i++
}
return $sitemap
} try{
    #set your domain and the array to hold the links
    $url = "www.domain.com"
    $sitemap = @{}     #read the page
    $testPage = Invoke-WebRequest "http://$url"
    #get all the links into the array
    $sitemap = get-URLlist -testPage $testPage -linkDepth 5
    $sitemap
}
catch{
    "There was an error- `r`n$($_.Exception.Message)"
}

I wrote this with an understanding of what the links look like on this page, so the function formats them with the given URL.