Tangential to our collaborations at Nikon Hacker, astronomer and I have cause to interface directly with Sony image sensors. It happens that these tend to be available second-hand for relatively cheap—likely pulled from cameras sold for repair. For example, a while back I snapped up a D800 sensor module for just USD $80! An amazing deal for a 36 megapixel full frame image sensor, assuming it works as advertised, but I have not tested it yet…

Thanks to reverse-engineering efforts on the Nikon firmware, we were able to begin mapping the register space of the Sony IMX071 sensor used in the Nikon D5100. Last I checked, there’s still a lot we don’t know; though that’s a topic for another post. In any case we can replay the register values used by the D5100 to get the sensor up and running independently, so I’ll consider that “solved” for now.

The current challenge is interfacing the LVDS pairs, which carry the actual image data. LVDS stands for Low-Voltage Differential Signalling. For those not familiar, differential signaling schemes transmit the signal as the difference between two complimentary signals. I defer to Wikipedia for the advantages of differential signals.

Observing the LVDS clock (green) and a data line (yellow) with the D5100 in Live View mode, we see the IMX071 is driving its 8 data outputs at 320Mbit/s, with a 160MHz DDR clock. That’s a grand total of 2.56Gbit/s of image data screaming out of the sensor. Fwoosh.

The probing technique for the above image is not ideal, nor is the scope’s 200MHz bandwidth. Still, it’s enough to sanity-check the data rate and prepare for phase two: sniffing the LVDS lines in-situ.

Somehow we need to capture a chunk of 2.56Gbit/s data for analysis. While we could prooobably take a bunch of scope captures blah blah blah, that sounded awfully boring. It just so happened that while cleaning my desk I ran into a yet-unused Cypress FX3 SuperSpeed Explorer Kit.

The Cypress FX3 has a nifty GPIF II interface. tl;dr it’s up to 32bits at up to 100MHz. It’s not capable of LVDS, so we need to convert that to single-ended. While we’re doing that, we also need to deserialize it: given our 8bit 320MHz input, we can maintain the same 2.56Gbit/s data rate with a 32bit 80MHz interface to the Cypress chip and still be within its limits. Then we can stream everything to a PC and deal with it in software.

Initially I looked at LVDS receivers, dedicated deserializer ICs, etc. Many were built for display buses like FPD-Link and might do the job here, but there was a lot of stuff to sort through. I got to thinking how handy it would be to have a general-purpose, reconfigurable LVDS thing hanging off the Cypress FX3 kit. It could be configured to interface with lots of interesting high speed stuff.

And then the choice was obvious: an FPGA. It can deal with data rates well in excess of what we see here, deserialize however is needed, and be configured to interface to other sensors in the future.

Over the weekend I whipped up a board with a Xilinx Spartan-6 LX4 FPGA, some pads for directly soldering short jumper wires to the DUT, connectors for expansion, and of course the Cypress FX3 Explorer Kit headers.

Another benefit of using an FPGA can be seen here: there is no need to length-match the signals. Every IO on the Spartan-6 contains a delay line which can compensate for up to 1 UI (one bit time) of mismatch. At 320Mbit/s, each bit spends 3.125ns on the wire. Multiplying by the typical signal velocity on FR4 of 6 inches/ns, each bit takes up a whopping 18.75 inches! So with the IODELAY2 feature we can compensate for up to 18.75 inches of mismatch, no problem 😀

It is worth noting, however, that these IO delay lines are more commonly used to center the data eye on the clock edge to meet setup and hold timings with a healthy margin. For more information, check out the Spartan-6 SelectIO Resources User Guide.

Anyway, after several hours of careful PCB layout and SI simulation I sent the design off to the cheapest quick-turn Chinese fab I could find…

If all goes well the next post will cover the protocol as seen by the sniffer.