Should I get a NAS?

Most recent language models have switched from the typical ReLU and GELU activation functions to something called SwiGLU. In trying to understand it, I realized most materials assume a lot of prior knowledge and there isn't a simple explanation of why it is good. So let's go the bottom-up approach and try to explain it from scratch. Activation...

Recently my posts were getting a bit too techy and serious, so I want to make up for it. Last year, I was in charge of setting up and maintaining our 40+ camp bikes, trikes, e-bikes and scooters, so here is a list of advice I gathered for anyone who needs to do some maintenance on theirs. Do it camp-wise Being able to fix the main problems for...

Coding time-related stuff is complicated: timezones, concurrency, API quotas, multi-party synchronization, networking… everything that time touches becomes messy. In this post, I want to describe some typical time-related function wrappers that are useful during the dev experience, with a special focus on front-end examples. Throttle Throttling...

After so long, I still find basic stuff in Python that I didn't know about. Here are some of my most recent TILs. --> You can use underscores to separate digits in a number and they'll be ignored by the interpreter: >>> 1000000 # confusing 1000000 >>> 1e6 # not confusing but creates a float 1000000.0 >>> 1_000_000 # great! 1000000 The sum...

After so long, I still find basic stuff in Python that I didn't know about. Here are some of my most recent TILs. --> You can use underscores to separate digits in a number and they'll be ignored by the interpreter: >>> 1000000 # confusing 1000000 >>> 1e6 # not confusing but creates a float 1000000.0 >>> 1_000_000 # great! 1000000 The sum...