philip’s curve - Home

Gradient boosting?

Yet another article trying to explain how GBMs work.

There’s no shortage of machine learning hot takes on how the XGBoost gradient boosting framework continues to defy the no-free-lunch theorem by being perhaps the most versatile learning…

Nested cross-validation

A solution for overfitting in model selection

In Why do cross-validation, I described cross-validation as a way of evaluating your modeling workflow from start to end to help you pick the…

Why do cross-validation?

It can be difficult to see the forest for the trees sometimes. And it certainly can be the case with cross-validation.

In short, we do cross-validation to pick the right model. But what does that mean for our ‘final model’?

WSL2 and R

Why and how to use R from WSL2.

R

Windows Subsystem for Linux (WSL) is a…

Charts can lie

It’s not cool if your chart’s designed to mislead, YB.

Data visualization is something we use to understand our data. Sure, it has some aesthetic function, but its primary purpose is to communicate what’s in the data.

Learning to sell data science

Half of your job, regardless of what that job is, is being able to sell your work.

Half of your job, regardless of what that job is, is being able to sell your work. (source)

Link functions matter

Benchmarking different link functions

There’s a bit of a conversation on Twitter whether to use linear models or logistic models for estimating effect sizes in the case of binary response variables. I…

Reflections on the tidyverse

Why I like (and use) the tidyverse

R

tidyverse

File paths and data science projects

Some thoughts on keeping a data science project tidy.

The Australian gender wage gap: interactive viz

A little more dataviz

ggplot2

wage-gap

I wrote a piece at The Ambitious Economist on Australia’s gender wage gap with some good ol’ ggplot2 dataviz. However, there was a chart showing the gender wage gap by occupation that could have benefited from some interactive visualisation. Fortunately, the…

PSA: Python, OLS and perfectly collinear variables

Unlike most implementations of linear models, Python packages don’t usually drop perfectly collinear variables.

econometrics

python

Unlike most implementations of linear models (e.g. Stata, R), Python packages don’t usually drop perfectly collinear variables.

My first package: bnmr

A wrapper to access Malaysia’s central bank’s API

R

bnmr is a wrapper for httr to access BNM (Malaysia’s central bank)’s recently released API. I wrote this to practice writing R packages, with lots of help from the R Packages boo…

Exploring GE14 results with Python (part 2)

Can we use machine learning to understand the demographic factors underlying the election results?

Python

See here for part 1.

Exploring GE14 results with Python (part 1)

Some exploratory insights from the 14th Malaysian general election

Python

There’s an interesting blog post up on DataTarik about using random forests to model the outcomes of Malaysia’s 14th general election…

Visualising government expenditure data with ggplot2

Visualising Malaysia’s stagnating development expenditure

R

I thought I’d practice my rusty R skills when I came across a post by Khairil Yusof on the Sinar Project Facebook group that Malaysian Administrative Modernisation and…