More examples of fasthttp Client usage?

[marcoroganovic]

Hi,

Being new to Go lang, it would be much appreciated if this library had more examples of using HTTP client in a right way.

The only way I've found so far was from blog post found on internet.

func doRequest(url string) {
        req := fasthttp.AcquireRequest()
        req.SetRequestURI(url)

        resp := fasthttp.AcquireResponse()
        client := &fasthttp.Client{}
        client.Do(req, resp)

        bodyBytes := resp.Body()
        println(string(bodyBytes))
        // User-Agent: fasthttp
        // Body:
}

But I don't know if this is optimal resource wise, since I need to make really huge number of GET and POST requests, and Go seems like good language for that.

[dgrr]

Try to allocate only the needed resources. In this case try to call *Release functions.
And do not make a new client every time you call doRequest, try to parse the client as a parameter to the function or just use the fasthttp.Do call

func doRequest(url string) {
        req := fasthttp.AcquireRequest()
        resp := fasthttp.AcquireResponse()
        defer fasthttp.ReleaseRequest(req) // <- do not forget to release
        defer fasthttp.ReleaseResponse(resp) // <- do not forget to release

        req.SetRequestURI(url)

        fasthttp.Do(req, resp)

        bodyBytes := resp.Body()
        println(string(bodyBytes))
        // User-Agent: fasthttp
        // Body:
}

[9mm]

id also be curious about this. I'm new to go and I need to make hundreds of thousands of outgoing HTTP requests per second. I started out doing this in elixir but its really having difficulty there is some bottleneck that myself and a few people smarter than me dont know how to solve

I thought I would just try learning/writing it in go using fasthttp and see if that helps mitigate the issues

[erikdubbelboer]

@9mm what kind of examples would you like? The example @dgrr posted above is the basic Client usage.

[marcoroganovic]

Hi @erikdubbelboer if I may also give my opinion on what examples would be useful. I've lately started working at low latency, high throughput projects, started with Node.js, but quickly saw some limits and switched to Go language, quickly saw that there are also some limits unless you tweak some OS configuration, bunch of benchmarks are only making hello world to localhost:8080, toy examples, there is no resources that explain network programming, in Go for instance, in a practical way.

If you can point me at some, I would really appreciate it.

In description of this library there is bunch of explanations about theory (which I really like), but not much code samples, and to be honest, being new to Go it's hard for me to start being productive, since I'm always thinking if approach that I took is good enough, especially since I've tried to implement something with net/http and fasthttp and got worse performance with fasthttp library. Probably, because of misuse, and I think a lot of people experience similar issue.

[9mm]

@marcoroganovic what is your particular use case out of curiosity, and what languages did you already try?

I am building an API that needs to responds to 10's of millions of requests per day, all while making an outgoing HTTP request, once it gets that response, i send that data in the response back to the initial request.

So it's like:

request -> make 2nd request -> get 2nd request response -> send response

I originally tried Elixir and I tweaked every linux setting known to man but couldnt seem to get the server to stop hitting this invisible bottle neck. CPU usage AND memory would not be even close to 100% but the connections would just start dropping in huge chunks (like 80% of connections would just die).

I started a thread here and also tried every suggestion. I recommend looking at some of the links in this thread, it answers what youre asking, and its not just related to elixir, its basically networking level settings: https://elixirforum.com/t/difficult-debugging-problem/18988

I dont really know what to try next, so I was hoping just using go might solve some issues... but overall i am going in fairly blind

[9mm]

@marcoroganovic heres one good link from the thread if you dont want to sift through it https://www.rabbitmq.com/networking.html#dealing-with-high-connection-churn

[9mm]

And this guy links to a ton of helpful links in his reply, but most seem to be covered by the link in the rabbitmq link in my previous reply https://elixirforum.com/t/difficult-debugging-problem/18988/90

Sadly.................... I tried all this and I still faced an issue... but I am also not a networking guy so I could have done something wrong.

I'm just starting from scratch with Go and hoping it was some language ecosystem problem.

After I build my app in Go, which I'm doing now using fasthttp... I'm going to install it on a dedicated server without fancy kubernetes (which introduces all kinds of extra variables with DNS and host/bridge networks etc) and just trying to do it that way...

[9mm]

I would be curious about the worse performance with nethttp... ive heard that from several places online when researching which seemingly points to misuse. I'd be curious about that... maybe post your code and blank out the sensitive stuff and Erik could possibly give us his .02 cents if he feels up to it.

I myself am just learning go to begin with so I haven't even built my app yet, I was planning on doing that this upcoming week

[marcoroganovic]

Hi @9mm, I'm right now making 700 million outgoing requests per day, but I need to scale it to 3 billion, I can probably do it with throwing more servers at the problem, but I want to utilise potential of current instances to the max.

Thank you for the links, here is also useful link: https://www.http-kit.org/600k-concurrent-connection-http-kit.html

It's the only guide that showed that you need to increase open files limit, and also increase range of virtual network addresses.

[marcoroganovic]

Probably to people who have computer science background this is common sense, but for me it was hard in the beginning, and only broaden my research skills to operating system internals.

[dgrr]

So @marcoroganovic which examples do you need?

[marcoroganovic]

Currently, I'm using net/http with transport client that looks like this.

func initClient() {
	tr := &http.Transport{
		MaxIdleConns:        1024,
		MaxIdleConnsPerHost: 1024,
		TLSHandshakeTimeout: 10 * time.Second,
		Dial: (&net.Dialer{
			Timeout:   75 * time.Second,
			KeepAlive: 75 * time.Second,
		}).Dial,
	}

	client = &http.Client{Transport: tr}
}

And I got pretty good results, but I want more, but I don't have that good benchmarking skills, so fasthttp sounded to me like something that is especially good at the problem that I want to solve, calling multiple endpoints upon requests, collecting all responses, responding back to user in a timely manner.

[marcoroganovic]

Without wasting resources.

[dgrr]

The client automatically handles your connections with the servers. So if you open a connection with a host and you want to send more than one request this won't create one TCP connection per request and so on... So there is nothing similar to the net/http.Transport here. To do so you just need to use the same client for all the request you are going to perform. The best practice here it's to use client.Do function and reusing Request and Response every request. If you need more specific examples we can help you here but we'll need details.

[marcoroganovic]

Thank you @dgrr , I'll keep investigating, and testing. Can you maybe recommend me some resources to learn more about this?

[dgrr]

Well... I don't know so much about resources, just keep your eye on HTTP-related projects like https://github.com/valyala/tcplisten

[9mm]

@dgrr sorry dumb question, are you referring to keepalive? Or http/2 multiplexing... or something else?

Are you saying fasthttp opens and closes the connection for every request rather than leaving it open and using keepalive?

@marcoroganovic impressive volume, that basically is on par with what I was doing on the previous iteration of this project (which had 10-100x more requests and could have scaled to 1m+ per second). That project never worked out technically so the thing Im working on now is less volume, but i'm getting hints of all the same problems.

I have 2 questions for you...

1. how many servers do you have, and how much memory/cpu does each server have to handle 700m incoming?
2. have you found any help on how to best calculate # of workers in the connection pool, as well as keepalive timeout time? These are arbitrary guesses in my first round but i wanted to make better estimated guesses

[dgrr]

@9mm No, I said it keeps the connection open until one of the peers closes the connection. And I am not referring to anything related to HTTP/2. HTTP/1.1 supports request multiplexing.

[9mm]

@marcoroganovic are you by chance in ad tech working with XML feeds? lol

[marcoroganovic]

@9mm yes, I do, also I've read every comment on Elixir forum, really great thread, a lot of useful info, I've started to learn Elixir just before Go, and it seems like great language, but didn't dig that deep into it.

It's 700 million outgoing requests, so every request that I receive spins 7 go routines, I have three servers for this with only Go binary and htop installed.

As far as number of workers goes, it's arbitrary, I've posted client instance above.

But to be honest, this is poor performance, I have a lot of 408 HTTP error codes. Also, this my first project in this domain, usually working with Node.js and React, but want to learn some of these lower level things.

I really started to appreciate type system, even though in the beginning I had certain unfriendly feeling while working with it.

Also, have you maybe tried Rust, I've seen some impressive benchmarks that people had with it, and being that low-level, I think that syntax and type system are really approachable.

@dgrr can you maybe post example where you would call 10 different endpoints concurrently with fasthttp using minimal resources?

Is it just like the example that you've posted above, or you would do something else to support it?

[9mm]

I considered learning rust but I was stopped by the difficulty vs time I want to invest in this. I figure Go however should be extremely fast and perhaps close to rust. I think the outbound requests are whats really killing it, but I was never convinced it wasnt an OS level issue. I did try a ton of things, but never ultimately figured it out. Either way, I still think it was a bottleneck somewhere rather than "the language is too slow". Go seems extremely fast, I imagine at that scale things just need to be tuned much better than my current abilities

[9mm]

I do recall that doing 1-3 outgoing requests was OK, however more than 4-5 and it would absolutely obliterate performance.

I almost used this, but I do not know java. https://datakernel.io/docs/core/http.html

[erikdubbelboer]

Here is a simple example I just wrote that incorporates some of the ideas you will need to follow to get the most out of your server: https://gist.github.com/erikdubbelboer/fe4095419fca55e2c92b3d0432ccd7fc
Keep in mind that it's not only about fasthttp, you should also try to get the rest of your code as fast as possible! That means trying to reduce the allocations in the rest of your code as well.

Let me know if any of you need more help/examples.

[marcoroganovic]

Wow @erikdubbelboer thank you very much! I'll investigate every line of code, to be honest it contains a lot of things which are foreign to me yet.

[9mm]

@erikdubbelboer you are awesome -- thank you!!

[ArkhipovK]

How can I add some parameters to get request, except hardcode them into URI?

[erikdubbelboer]

Either add them to the string of your URL yes or use fasthttp.URI. For example: https://play.golang.org/p/ExAvAg3PnzX

[tsingson]

my repo https://github.com/tsingson/fasthttp-example for fasthttp client and server , FYI

[vinhjaxt]

Here is a simple example I just wrote that incorporates some of the ideas you will need to follow to get the most out of your server: https://gist.github.com/erikdubbelboer/fe4095419fca55e2c92b3d0432ccd7fc
Keep in mind that it's not only about fasthttp, you should also try to get the rest of your code as fast as possible! That means trying to reduce the allocations in the rest of your code as well.

Let me know if any of you need more help/examples.

Hello @erikdubbelboer, in your gist note, why did you declare the client but not use it?

[erikdubbelboer]

@vinhjaxt good find, it should be client.Do instead of fasthttp.Do. I have updated the gist.

[stokito]

#1208