Sparsely-gated Mixture Of Experts (MoE)

In transformer models, the attention block is typically followed by a feed forward layer (FF), which is a simple fully-connected NN with a hidden layer and nonlinearity. Here's the code for such a block that uses ReLU: def feed_forward_relu(x, W1, W2): """Feed-...

Learning Software Skills fast: what worked for us best in the last 15 years

Introduction In this episode, we discuss how to learn effectively as a software engineer. Why some people seem to learn faster than others? What are some practical ways to speed up your learning? Instead of promising magical shortcuts to becoming a principal engineer in months, w...

Some signal boosting for Go

#​550 — April 16, 2025 Unsub  |  Web Version 🥚 We're taking a little break for Easter but didn't want to take the entire week off, so we have a slimline issue for you today :-) Back to full speed next Wednesday!__Peter Cooper, your editor...

Scheduling In Go : Part II - Go Scheduler

This blogpost is the second installment in a three-part series exploring the mechanics and semantics of the Go scheduler. Despite being published in 2018, the content remains relevant today, as the Go scheduler’s design continues to influence the development of efficient an...

Go 1.24's "tool" directive

Go 1.24 added a new tool directive that makes it easier to manage your project’s tooling. I used to rely on Make targets to install and run tools like stringer, mockgen, and linters like gofumpt, goimports, staticcheck, and errcheck. Problem is, these installations were glo...

Series

Cross-entropy and KL divergence

Cross-entropy is widely used in modern ML to compute the loss for classification tasks. This post is a brief overview of the math behind it and a related concept called Kullback-Leibler (KL) divergence. Information content of a single random event We'll start with a single event...

Bobcoin, blockchains, and cryptocurrency

How do cryptocurrencies actually work, though? Join Alice and Bob as they embark on designing a new digital ledger for secure “Bobcoin” transactions.

Capturing console output in Go tests

Ideally, every function that writes to the stdout probably should ask for a io.Writer and write to it instead. However, it’s common to encounter functions like this: func frobnicate() { fmt.Println("do something") } This would be easier to test if frobnicate wou...

Automagically migrating `golangci-lint` configuration to v2 with Renovate

As mentioned in Using Renovate to manage updates to golangci-lint versions I very much recommend using Renovate to update your golangci-lint versions. With the release of golangci-lint v2 a couple of weeks ago, consumers need to perform a one-time migration of their configuration...