In transformer models, the
attention block
is typically followed by a feed forward layer (FF), which is a simple fully-connected
NN with a hidden layer and nonlinearity. Here's the code for such a block that
uses ReLU:
def feed_forward_relu(x, W1, W2):
"""Feed-...
Introduction
In this episode, we discuss how to learn effectively as a software engineer.
Why some people seem to learn faster than others?
What are some practical ways to speed up your learning?
Instead of promising magical shortcuts to becoming a principal engineer in months,
w...
#550 — April 16, 2025
Unsub | Web Version
🥚 We're taking a little break for Easter but didn't want to take the entire week off, so we have a slimline issue for you today :-) Back to full speed next Wednesday!__Peter Cooper, your editor...
This blogpost is the second installment in a three-part series exploring the mechanics and semantics of the Go scheduler. Despite being published in 2018, the content remains relevant today, as the Go scheduler’s design continues to influence the development of efficient an...
Go 1.24 added a new tool directive that makes it easier to manage your project’s tooling.
I used to rely on Make targets to install and run tools like stringer, mockgen, and
linters like gofumpt, goimports, staticcheck, and errcheck. Problem is, these
installations were glo...
Cross-entropy is widely used in modern ML to compute the loss for classification
tasks. This post is a brief overview of the math behind it and a related
concept called Kullback-Leibler (KL) divergence.
Information content of a single random event
We'll start with a single event...
How do cryptocurrencies actually work, though? Join Alice and Bob as they
embark on designing a new digital ledger for secure “Bobcoin” transactions.
Ideally, every function that writes to the stdout probably should ask for a io.Writer and
write to it instead. However, it’s common to encounter functions like this:
func frobnicate() {
fmt.Println("do something")
}
This would be easier to test if frobnicate wou...
As mentioned in Using Renovate to manage updates to golangci-lint versions I very much recommend using Renovate to update your golangci-lint versions.
With the release of golangci-lint v2 a couple of weeks ago, consumers need to perform a one-time migration of their configuration...