Szathmáry László honlapja @ DEIK | Nim2 / download a webpage

Download a webpage and get its content as a string.

import std/httpclient

proc get_page(url: string): string =
let client = newHttpClient()
try:
client.getContent(url)
except HttpRequestError as e:
stderr.writeLine("Error: ", e.msg)
""

proc main() =
let
url = "https://index.hu"
html = get_page(url)

echo "'" & html & "'"

# ##########

when isMainModule:
main()

SSL error

If you try to compile it, very likely you'll get an error:

Error: SSL support is not available. Cannot connect over SSL. Compile with -d:ssl to enable.

As the message says, pass the -d:ssl switch to the compiler:

$ nim c -d:ssl main.nim

Or, create a file called config.nims with the following content:

switch("define", "ssl")

Place it next to the source code:

$ tree .
.
├── config.nims
└── main.nim

Now, when you compile your program, the nim compiler will automatically read this config.nims file. Thus, simply compile the program like this:

$ nim c main.nim

Retry if there's a network error

Downloading a webpage may fail for various reasons. It can be a good idea to retry the download a few times before giving up:

import std/os # sleep
import std/httpclient

# delay: in msec
proc get_page(url: string, retries=10, delay=200): string =
let client = newHttpClient()
for attempt in 1 .. retries:
try:
return client.getContent(url)
except Exception as e:
stderr.writeLine(&"Attempt {attempt + 1} failed: {e.msg}")
sleep(delay) # msec
# endfor
""