Background
A few days ago, I was revisiting one of my side projects that I haven’t gotten the chance to complete.
(Don’t we all have side projects that we never got to finish? 😅)
The project is a CLI tool written in Go that consumes WakaTime API to get stats and render them in the terminal.
The idea of the project is to render the stats in a line or two so that you can use the output of the program in Tmux, Polybar or other tools for customizing your setup. There might already be a project doing that similar thing out there, but that’s not the point of this blog post.
It was still in the very early stage when I left it, but I remember that the authentication process was working before. Upon trying to run the commands I got an authentication error, but this was expected since I haven’t implemented refreshing of the tokens yet, when I ran the login command however I was surprised by this error.
$ ./wakago login
2023/08/04 01:33:12 Waiting for redirect URI
2023/08/04 01:33:12 Server started
2023/08/04 01:33:15 Exchange: invalid character 'a' looking for beginning of value
2023/08/04 01:33:15 http: panic serving 127.0.0.1:51662: runtime error: invalid memory address or nil pointer dereference
goroutine 6 [running]:It seems that the error was happening during the “exchange” token phase, this is also known as the “Token Request” stage.
For WakaTime the URL is https://wakatime.com/oauth/token.
You can read more about WakaTime Authentication in their official documentation. For this project, I used the official Oauth2 library from Golang.
Debugging
I initially did a lot of things wrong upon looking at this error/bug in retrospect. I wasted my time looking into the environment variables, and different fields of the Oauth2 config, and also looked into the ‘state’ that was being generated for the authorization stage. In hindsight, I could’ve just started looking down on the exact code that throw the error.
Out of frustration, I tried out the process of authenticating using the given example in Python, to rule out if the issue was from the WakaTime side.
I initially go to the definition of the Exchange method of the oauth2.Config struct but since there were a lot of things happening under the hood I didn’t figure it out at first glance.
I now tried using the debugger and attached some break points in the Oauth2 code itself to see what’s happening.
Aha!
Upon closer inspection, it seems that I’ve found the bug! The Content-Type in the response header from WakaTime API is set to text/html, which doesn’t fall into the
first case in the switch statement from the Oauth2 module. Upon inspecting the actual response body I now know that the format of the actual response body looks like a URL-encoded string. It really should’ve been handled by the first switch case.
Here json.Marshal() tried to unmarshal the text but it’s not a valid JSON, which causes the program to return the error.
This is where the cryptic: invalid character 'a' looking for beginning of value came from.
In reality, JSONs should typically start with a { or [ character.
Workaround
Since I wanted to see if I can get this thing fixed by myself, I decided to fork the Oauth2 module from Google and just add a new case there to handle the text/html Content-Type.
So the token.go code would look something like this:
func doTokenRoundTrip(ctx context.Context, req *http.Request) (*Token, error) {
// ...
var token *Token
content, _, _ := mime.ParseMediaType(r.Header.Get("Content-Type"))
switch content {
// case "application/x-www-form-urlencoded", "text/plain",
case "application/x-www-form-urlencoded", "text/plain", "text/html":
// ...The workaround code looks straightforward enough.
But making it work is not as simple as I thought it would be. Since Go modules are typically named with domain names similar to Java packages I also need to update all instances golang.org/x/oauth2 to github.com/laureanray/oauth2. [See this commit]
NOTE: I have to update all the instances since we can’t invoke any code from the ‘internal’ package in Go. This means I can’t just update the top level module definition and use the
golang.org/xoauth2/internalpackage. For more information see this design doc.
After that, I updated my dependency by doing:
go get github.com/laureanray/oauth2@5703a4d3f486f60438989613c11bf179ca8494ddVoila! It worked!
$ ./wakago login
2023/08/05 02:00:14 state: e8efa1de9879fc39653d411001b4e91d31254d47
2023/08/05 02:00:14 Waiting for redirect URI
2023/08/05 02:00:14 Server started
2023/08/05 02:00:16 Trying to exchange: 7bB0qxDkcggI4CTyFn4NUAr4gZglkgEIsXCTKq9iKEPzAE9B0t2TgPWFvYG3QFNyAp7gFOUNp7xVGpOO
2023/08/05 02:00:17 89 bytes written
Lessons Learned
The Content-Type seems so trivial, but API Developers must take HTTP headers seriously, there’s a reason that it’s there in the first place. I’m not throwing any shades at WakaTime dev here, people make mistakes, and I occasionally ignore this header too when playing around with APIs because I always know what response I’m getting. Sometimes I also don’t pay attention to this header value that much when building REST APIs, I usually assume that the client knows what kind of response it’s getting and how to parse it.
I think that’s also the beauty of protocols and standards, if we would all follow and implement them properly software should work as designed and intended.
PS.
I already reached out to WakaTime on X (formerly Twitter) regarding this bug, and they said it’s been fixed now.
Overall this was a nice experience and I learned a lot. And a huge shoutout to WakaTime
Thanks for reading. <3