Floki: Requests-like HTTP Client powered by libcurl

Community Showcase
1

Hey Everyone! I’ve just released a new version of Floki which is a straightforward HTTP client with an API similar to the Python requests package. It’s powered by libcurl, so if you want to use the bindings I wrote for this directly, you can find them at my mojo-curl repository.

Maybe feature parity will be achieved one day, but we’ll see where this goes.

Unfortunately, due to libcurl’s client interface making use of C variadics, this package requires the use of a small shim c library to be able to call curl_easy_setopt. So, you can’t just link libcurl and be good to go. You’ll need to make use of the curl_wrapper subpackage through the instructions in the readme!

2

The underlying libcurl bindings have been updated to support Mojo 1.0.0b1! Floki will follow soon.

I’ve converted the simple examples provided in the curl documentation into running Mojo examples! Well, most of them at least. Some examples rely on local configuration, like serving something over a unix socket or a web server running on localhost.

The examples can be found here: mojo-curl/examples/curl/api at main · thatstoasty/mojo-curl · GitHub

Aside from that, I went through and cleaned up a bunch of half baked and redundant code. Eventually, I’d like to break the c submodule into it’s own package once C FFI is stable so that way there’s a raw bindings package and an opinionated user friendly interface as it’s own package.

3

Floki has been updated to support Mojo 1.0.0b1! Please check out v0.3.2: Release v0.3.2 · thatstoasty/floki · GitHub

Aside from some bug fixes, the main feature that I’ve added is adding struct deserialization/serialization for JSON request data and response bodies. To say I implemented it might be a stretch, since it heavily leans on emberjson’s struct serialization/deserialization :slight_smile:.

Response Body Deserialization

@fieldwise_init
struct Todo(Movable, Defaultable, ImplicitlyDestructible, Equatable, Writable):
    var userId: Int
    var id: Int
    var title: String
    var completed: Bool

    def __init__(out self):
        self.userId = 0
        self.id = 0
        self.title = ""
        self.completed = False
    

def main() raises -> None:
    var response = Session().get("https://jsonplaceholder.typicode.com/todos/1")
    assert_equal(response.status, Status.OK)

    var todo = response.body.as_json[Todo]()
    assert_equal(todo.userId, 1)
    assert_equal(todo.id, 1)
    assert_equal(todo.title, "delectus aut autem")
    assert_equal(todo.completed, False)

Request Data Serialization

from floki.session import Session

@fieldwise_init
struct Point:
    var x: Int
    var y: Int

def main() raises:
    var session = Session()
    var r = session.post("https://httpbin.org/post", data=Point(0, 1))
4

Floki has been updated to support Mojo 1.0.0b2 and a few new Session configurations have been added. It’s really a first pass at this, I’m exploring options for the user interface to keep the number of ways you can configure options tight and focused, but also making it not so verbose. I’ll continue to iterate on it.

Timeouts

Configure granular connect/total timeouts (in seconds) on a Session. A bare number
is treated as the total timeout:

from floki.session import Session
from floki.timeout import Timeout

def main() raises -> None:
    var session = Session(timeout=Timeout(connect=5.0, total=30.0))
    var quick = Session(timeout=10)  # 10 second total timeout
    var r = session.get("https://example.com")

Retries with backoff

Retry failed transfers and retryable status codes with exponential backoff:

from floki.session import Session
from floki.retry import Retry

def main() raises -> None:
    var session = Session(
        retry=Retry(max_retries=3, backoff_factor=0.5, status_forcelist=[500, 502, 503, 504])
    )
    var r = session.get("https://example.com")

The delay before retry n is backoff_factor * 2 ** (n - 1) seconds.

Proxies

from floki.session import Session
from floki.proxy import Proxy

def main() raises -> None:
    var session = Session(proxy=Proxy("http://proxy.example:8080"))

    # With credentials and a bypass list:
    var authed = Session(
        proxy=Proxy(
            "http://proxy.example:8080",
            username="user",
            password="secret",
            no_proxy="localhost,127.0.0.1",
        )
    )
    var r = session.get("https://example.com")

TLS verification

TLS certificate and hostname verification is enabled by default. It can be disabled
(use with great caution) or pointed at a custom certificate authority bundle:

from std.pathlib import Path
from floki.session import Session
from floki.tls import TLS

def main() raises -> None:
    # Disable verification — only for testing or trusted networks.
    var insecure = Session(tls=TLS(verify=False))

    # Use a custom CA bundle (e.g. a private PKI or self-signed cert).
    var custom = Session(tls=TLS(ca_bundle=Path("/path/to/ca-bundle.pem")))
    var r = custom.get("https://internal.example.com")

timeout, retry, proxy, and tls are also accepted directly by the free
functions (e.g. floki.get(url, timeout=10)), which forward them to the one-shot
Session they create internally.

Examples

I’ve also added two examples of a server and client, using Flare for the server :slight_smile:. The todo example also includes usage of my Slight sqlite3 package to include a typical CRUD HTTP GET example of reading from a db and serializing the resource to a json response.