For the final (for now) part of this series, I am going to extend the particle filter to do online parameter estimation using online Expectation-Maximization (EM) to calculate an estimate of the autoregression parameter at each stage of the particle filter. There are many options for online parameter estimation, including…

# Hadoop Part III: Multiple Output in Hadoop

The output of the Hadoop MapReduce particle filter from the previous post is simply a list of doubles giving the state for each particle after resampling. This is not ideal because this post-resampling particle collection is a more crude representation of the post-observation state posterior than the pre-resampling, weighted collection. Obviously…

# Hadoop Part II: Particle Filters in Hadoop MapReduce

In this article, I’m going to go about implementing a basic particle filter in Hadoop MapReduce. This is really just a personal interest project for me to get started learning Hadoop based on an algorithm that I am familiar with and suits MapReduce (to some extent), but this might have…

# Hadoop Part I: Configuring Hadoop (for Complete Beginners)

When starting out with new technology, I often find that one of the most challenging bits is getting things into a place where I can start to write code. I usually find this process quite frustrating and it isn’t a part that I particularly enjoy. Now, this may say something…

# Hadoop Introduction: Using Hadoop for Sequential Monte Carlo (Particle Filter) Parameter Estimation

Recently, I have become interested in parameter estimation for sequential Monte Carlo (SMC) methods (primarily, but not exclusively, particle filters), due to writing about them for my forthcoming book (obligatory plug – sorry). And I’ve also become interested in learning about Hadoop (an interest not uncorrelated to my current job…