Stop Ship Issue and WoofLinks
Sup people, last Friday while shipping a batch of FloJacks we started seeing some unusual behavior and a few dead batteries. Our quality control procedure is to test out each FloJack with a standard set of 8 tags (all different shapes, sizes, and types) before boxing them up. Some FloJacks failed with the larger tags but a couple just had dead batteries. The dead batteries were very troubling since we have been using production FloJacks ourselves for over two weeks without issues. We spent the weekend taking apart FloJacks and studying their behavior. We found that a simple compiler flag change from last week was causing the FloJack to constantly reset.
Breakdown
The FloJack has a microcontroller that handles talking to the NFC transceiver as well as to the Host unit (iOS device). Since there’s no display or keypad on the FloJack, there’s not a lot of ways the little guy can tell us when things go wrong (like a reset). We have a couple of LEDs that give us some feedback but because we use them for so much, you can easily mistake one thing for another. The FloJack’s microcontroller firmware has been in development for over 2 years now (soooooo long!), but we’ve always compiled it with debug info. Compiling with debug info just means that there’s a build flag that triggers the inclusion a bunch of “extra stuff” into the FloJack’s firmware. This stuff makes it easier to debug issues. Since we were moving beyond the debug phase, we (I) thought it was wise to remove that debug build flag and save on fluff. Leaner firmware translates to less power consumption (our biggest hurdle).
Turns out that changing this single compiler flag had a catastrophic result: constant resets. Removing the debug “extra stuff” changed the timing aspects of some state machines we have running. It’s like removing a few teeth in a gear box, it can wreak havoc in the whole system. Problems weren’t immediate or obvious since we do most of our testing right after inserting the battery in FloJack assembly. When the FloJack first gets powered on, it calibrates itself for about 10 secs. During this calibration sequence it pulses a ton of energy through the NFC transceiver to gauge what power decay values it senses in open air. It then uses these values to compute the sniffer threshold. The sniffer threshold is what tells the FloJack when a tag is present or not. That way the FloJack doesn’t expend valuable power trying to talk to tags that aren’t present.
Bringing it all together, removing the debug info ultimately locked up state machines causing slow but repetitive resets that continuously reran calibration and killed the battery over 2-3 days. This weekend we fixed the issue, but now have to disassemble all the FloJacks we built last week, replace the batteries, and reflash them with the fixed firmware. To the customers that we’ve already shipped out your unit, feel free to ship it back to us if you’re experiencing problems and we will replace it for you. Sorry for the inconvenience.
WoofLinks
On a more positive note, we wanted to give a shout out to one of our Kickstarter backers, Kirk Mower, who just launched an awesome project called WoofLinks. WoofLinks is a system that aims to bring NFC to every dog collar. Kirk hit us up when we first launched the FloJack project last November and we’re floored on how far he’s come with it. We even 3D printed one of Kirk’s first collar buckles with an NTAG203 tag inside to get the idea rolling. Now people and doggys can socialize in a whole new way… and with FloJack even with iOS devices too!!! Check out Kirk’s project, help get the word out, and push the NFC movement once step closer to mass adoption.
Thanks homies,
Richard and the Flomies.
Leave a Reply
Want to join the discussion?Feel free to contribute!