Rich Pang

https://rkp.science/ (RSS)
visit blog
An alternative construction of Shannon entropy
2 Aug 2024 | original ↗

TL;DR: Shannon’s entropy formula is usually justified by showing it satisfies key mathematical criteria, or by computing how much space is needed to encode a variable. But one can also construct Shannon’s formula starting purely from the simpler notion of entropy as a (logarithm of a) count—of how many different ways a distribution could have …...

Estimating slow correlations from samples of limited duration
8 Nov 2022 | original ↗

TL;DR: Many biological time-series have slow correlation timescales, but default estimators can greatly underestimate these correlations, even if you have enough data to resolve them. Normalizing by a triangle function removes the bias, but at the cost of potentially yielding an “invalid” correlation function estimate. Science is replete with...

Using regularization to handle correlated predictors
18 May 2022 | original ↗

TL;DR: Correlated predictors can yield huge instabilities when estimating weights in regression problems. Different regularization methods tame these instabilities in different ways. Regularization is usually introduced as a solution to overfitting. By penalizing certain parameter combinations it can improve how models generalize to held-out data...

↑ These items are from RSS. Visit the blog itself at https://rkp.science/ to find everything else and to appreciate author's digital home.