FingerIO 2D

Why this page needs a phone

A browser cannot open two microphones. getUserMedia hands out whatever the OS calls “the microphone”, and on iOS that is a single processed stream — so the second ellipse never exists. The companion iOS app opens the built-in array natively, plays the OFDM symbol from the speaker, and streams both raw channels here over a WebSocket. The phone does no signal processing: it emits, captures, and ships samples. Everything below runs in this page, against the same fingerio-dsp.js the single-microphone page uses.

Each packet carries the absolute sample index of its first frame. That is what makes streaming safe here where a WebRTC audio track would not be: a jitter buffer that quietly inserts one sample moves a reflector 3.6 cm, but an explicit index turns any discontinuity into a visible gap that forces a resynchronisation instead of a silently wrong position.

Connect

The address the app shows on its Bridge tab. This page must be served over http:// — a page loaded over https cannot open a plain ws:// socket, and the phone has no certificate.

Not connected

Geometry

The solver needs to know where the microphones sit relative to the speaker, measured along the phone’s long axis. Positive is towards the top of the phone. The Probe tab’s best-lag readout, converted to centimetres of air, gives you the spacing between the two mics; the speaker offset you can measure with a ruler.

Which streamed channel is which physical microphone is not something iOS tells us, so if the trace comes out mirrored along the phone’s axis, tick Swap.

The maths

Put the speaker at the origin and both microphones on the phone’s long axis at signed offsets m1 and m2. A finger at F = (x, y) at distance r = |F| from the speaker produces a total path length d = r + |F − M|. Expanding |F − M|² = (d − r)² gives

y m − r d = (m² − d²) / 2

which is linear in the two unknowns y and r. Two microphones make a 2×2 system, and x follows from r² = x² + y². No iteration and no initial guess — but note the sign of x is genuinely unrecoverable, because the whole arrangement is mirror-symmetric about the phone’s axis. That is the ambiguity the paper resolves by assuming you work on one side of the device, and it is why a third microphone is the real fix.

Delays are measured from each channel’s own direct path — the speaker heard directly by that microphone — so the total path is di = |mi| + Δic/fs and no clock offset between the speaker and the microphones ever enters the answer. Both arrivals are located the same way — a parabola fitted through the peak bin and its neighbours — so the shape of the band-limited pulse cancels in the difference instead of biasing every distance.

One detail cost most of the accuracy in simulation. The paper subtracts consecutive echo profiles to isolate the finger, and that works well for deciding whether something moved. But measuring the delay on that difference drags it late: two copies of a 12-sample-wide pulse only a few samples apart overlap, and subtracting one eats the leading half of the other, in proportion to how fast the finger is moving. Measuring instead against a slow clutter estimate — slow enough that a moving finger never enters it — took the simulated median error from 26 mm to 9.5 mm.