James Murphy

Intentionality Prediction

Intentionality Prediction

I’ve recently finished and submitted a paper on the interesting topic of intentionality prediction, that is, figuring out the intent of something from the action that it is taking.  In the paper (one of a number that I have published in this area), we look at how we can infer the intended destination of a tracked object using only its partial track observed up to the current time (which might only be observed via noisy or incomplete observations of, for example, its position).

The method using Bayesian statistics (in particular Bayesian tracking algorithms) to make probabilistic predictions about the tracked object’s intentions.  I can’t too much about the details of the method, since there might be a patent in the works, but I can at least post a couple of videos showing some of the results.  Firstly for destination prediction, and secondly for enhanced prediction of the object’s future state.

Some of our earlier work in this area (for predicting where a user is pointing on an interface in automotive applications) has been featured in Wired (second video, on YouTube here)!

Destination Inference

Below is a video demonstrating destination (and arrival time) inference using our algorithm.  There’s a lot going on, I’m afraid, and the video is rather small, so you might have to put it in fullscreen mode to see the details.  Sorry.  The central map shows a the track (blue line) of a ship (blue dot) heading to one of six possible ports (red dots) at some unknown time in the future.  The circle surrounding each port is indicative of how likely the inference algorithm thinks that port is as the target’s destination.  The graph on the right shows this as a bar chart of these posterior probabilities for each destination, with the correct eventual destination (port 2) highlighted.

The two graphs on the left relate to predicting the arrival time of the tracked object at its (unknown) destination.  The upper graph shows the estimated posterior distribution of the arrival time (with the red line showing the true arrival time and the green line showing the time now).  The lower graph shows the predicted arrival time (plus/minus one standard deviation) after each observation.

As you can see from the video, the algorithm is able to predict the correct destination for a large portion of the track.  Perhaps even more interesting is that the algorithm can also fairly accurately predict the arrival time of the object.  You can see from the lower left chart how its confidence increases as more observations become available, reflecting a concentration of the arrival time’s posterior density as more observations are made.

Enhanced Prediction

Another thing destination inference allows is an enhanced ability to predict where a tracked object is going.  The videos in this section demonstrate how our algorithm can be used to enhance predictions of the tracked object’s future position.

In these, the green track shows the trajectory that has been observed (noisily) up to the current time – this is all that we are basing the predictions on.  The red dotted track shows the actual future path of the object, with several steps ahead shown for each update of the observed track.  The grey shading shows the posterior predictive distribution of the object’s position at points in the future.  What we would like to see ideally is a dark area surrounding the red dot, which would mean that the prediction closely matches the future movement of the object.

There are two prediction videos below, one assuming we know the arrival time of the object at its destination (which simplifies things and makes the predictions more precise, but is not very realistic in a lot of settings), and one in which the arrival time of the object is unknown.

With a known arrival time:

Here, you can see that at first, because the algorithm doesn’t know where the object is headed (other than that it is going to one of the six ports), there are blobs of prediction density heading toward each port.  As the algorithm becomes more confident about the destination, these decrease in intensity for all but the true destination.

Without known arrival times, the situation is similar:

Here, however, the algorithm’s uncertainty about the arrival time is reflected in the ‘fingers’ of probability density heading towards each destination – the algorithm doesn’t (at first) know how far along in the approach to each possible destination it is.  As more observations become available, the algorithm becomes steadily more confident and the posterior prediction becomes much more tightly concentrated.


The paper is undergoing peer review, so we hope it will be published fairly soon.  Details to follow…