Throttle and debounce

Most recent language models have switched from the typical ReLU and GELU activation functions to something called SwiGLU. In trying to understand it, I realized most materials assume a lot of prior knowledge and there isn't a simple explanation of why it is good. So let's go the bottom-up approach and try to explain it from scratch. Activation...

Recently my posts were getting a bit too techy and serious, so I want to make up for it. Last year, I was in charge of setting up and maintaining our 40+ camp bikes, trikes, e-bikes and scooters, so here is a list of advice I gathered for anyone who needs to do some maintenance on theirs. Do it camp-wise Being able to fix the main problems for...

After so long, I still find basic stuff in Python that I didn't know about. Here are some of my most recent TILs. --> You can use underscores to separate digits in a number and they'll be ignored by the interpreter: >>> 1000000 # confusing 1000000 >>> 1e6 # not confusing but creates a float 1000000.0 >>> 1_000_000 # great! 1000000 The sum...

After so long, I still find basic stuff in Python that I didn't know about. Here are some of my most recent TILs. --> You can use underscores to separate digits in a number and they'll be ignored by the interpreter: >>> 1000000 # confusing 1000000 >>> 1e6 # not confusing but creates a float 1000000.0 >>> 1_000_000 # great! 1000000 The sum...

Developers spend at least one third of their time working on the so-called technical debt [1,2,3]. It would make sense then that non-technical people working with developers had a good understanding of an activity that takes so much of their time. However, this abstract concept remains elusive for many. I believe this problem has to do with the...