Data Science #13 - Kolmogorov complexity paper review (1965) - Part 2

Data Science #13 - Kolmogorov complexity paper review (1965) - Part 2

In the 14th episode we review the second part of Kolmogorov's seminal paper: Three approaches to the quantitative definition of information’." Problems of information transmission 1.1 (1965): 1-7. The paper introduces algorithmic complexity (or Kolmogorov complexity), which measures the amount of information in an object based on the length of the shortest program that can describe it.

This shifts focus from Shannon entropy, which measures uncertainty probabilistically, to understanding the complexity of structured objects.


Kolmogorov argues that systems like texts or biological data, governed by rules and patterns, are better analyzed by their compressibility—how efficiently they can be described—rather than by random probabilistic models. In modern data science and AI, these ideas are crucial. Machine learning models, like neural networks, aim to compress data into efficient representations to generalize and predict. Kolmogorov complexity underpins the idea of minimizing model complexity while preserving key information, which is essential for preventing overfitting and improving generalization.


In AI, tasks such as text generation and data compression directly apply Kolmogorov's concept of finding the most compact representation, making his work foundational for building efficient, powerful models. This is part 2 out of 2 episodes covering this paper (the first one is in Episode 12).

Det här avsnittet är hämtat från ett öppet RSS-flöde och publiceras inte av Podme. Det kan innehålla reklam.

Avsnitt(33)

 Data Science #34 - The deep learning original paper review, Hinton, Rumelhard & Williams (1985)

Data Science #34 - The deep learning original paper review, Hinton, Rumelhard & Williams (1985)

On the 34th episode, we review the 1986 paper, "Learning representations by back-propagating errors" , which was pivotal because it provided a clear, generalized framework for training neural networks...

23 Nov 202546min

Data Science #33 - The Backpropagation method, Paul Werbos (1980)

Data Science #33 - The Backpropagation method, Paul Werbos (1980)

On the 33rd episdoe we review Paul Werbos’s “Applications of Advances in Nonlinear Sensitivity Analysis” which presents efficient methods for computing derivatives in nonlinear systems, drastically re...

3 Nov 202557min

 Data Science #32 - A Markovian Decision Process, Richard Bellman (1957)

Data Science #32 - A Markovian Decision Process, Richard Bellman (1957)

We reviewed Richard Bellman’s “A Markovian Decision Process” (1957), which introduced a mathematical framework for sequential decision-making under uncertainty. By connecting recurrence relations to M...

19 Sep 202546min

 Data Science #31 - Correlation and causation (1921), Wright Sewall

Data Science #31 - Correlation and causation (1921), Wright Sewall

On the 31st episode of the podcast, we add Liron to the team, we review a gem from 1921, where Sewall Wright introduced path analysis, mapping hypothesized causal arrows into simple diagrams and provi...

26 Juli 202548min

Data Science #30 - The Bootstrap Method (1977)

Data Science #30 - The Bootstrap Method (1977)

In the 30th episode we review the the bootstrap, method which was introduced by Bradley Efron in 1979, is a non-parametric resampling technique that approximates a statistic’s sampling distribution by...

30 Maj 202541min

Data Science #29 - The Chi-square automatic interaction detection(CHAID) algorithm (1979)

Data Science #29 - The Chi-square automatic interaction detection(CHAID) algorithm (1979)

In the 29th episode, we go over the 1979 paper by Gordon Vivian Kass that introduced the CHAID algorithm.CHAID (Chi-squared Automatic Interaction Detection) is a tree-based partitioning method introdu...

23 Maj 202541min

Data Science #28 - The Bloom filter algorithm

Data Science #28 - The Bloom filter algorithm

In the 28th episode, we go over Burton Bloom's Bloom filter from 1970, a groundbreaking data structure that enables fast, space-efficient set membership checks by allowing a small, controllable rate o...

23 Maj 202539min

Data Science #27 - The History of Least Squares (1877)

Data Science #27 - The History of Least Squares (1877)

Mansfield Merriman's 1877 paper traces the historical development of the Method of Least Squares, crediting Legendre (1805) for introducing the method, Adrain (1808) for the first formal probabilistic...

2 Apr 202532min

Populärt inom Vetenskap

p3-dystopia
dumma-manniskor
allt-du-velat-veta
hacka-livet
ufo-sverige
rss-vetenskapsradion
rss-kriminologerna
svd-nyhetsartiklar
bildningspodden
det-morka-psyket
rss-vetenskapsradion-2
halsorevolutionen
ufo-sverige-2
medicinvetarna
dumforklarat
vetenskapsradion
sexet
rss-odla
barnpsykologerna
paranormalt-med-caroline-giertz