ENG | Arduino, Simple EEPROM Data Logger

Introduction

In this article, we’ll explore how (not) to build a simple data logger using an Arduino board, a BME280 sensor, a DS3231 Real-Time Clock (RTC) module, and an external EEPROM chip. This data logger will periodically record temperature and pressure measurements along with the current time, and store the data in the EEPROM for later retrieval, using code example from the previous article.

There’s nothing special about it: it should be very simple.

True data logger will need battery, monitoring battery voltage, possibly shut itself down when battery is low without restarting itself and EEPROM capacity in this example is too low.

But it’s just a starting point. Let’s go.

Code

//! \file   logger-20240323a.ino
//! \brief  Simple Arduino Data Logger (unoptimized)
//! \date   2024-03-23
//! \author Pavel Perina

// Based on
// https://github.com/pavel-perina/arduino-bmp280-minimal/blob/main/bmp280_arduino_reader/bmp280_minimal.ino
// https://www.pavelp.cz/posts/eng-arduino-rtc/
// https://www.pavelp.cz/posts/eng-arduino-eeprom/

#include <Wire.h>

// Set to 1 for fast test run with a few measurements and short intervals
// Overwrites EEPROM immediately
#define TEST_RUN 0

// Set frequency to 5 minutes (example)
constexpr uint8_t       frequencyMins   = 10;
#if !TEST_RUN
constexpr unsigned long frequencyMs     = 1000LL * 60 * frequencyMins;
#else
constexpr unsigned long frequencyMs     = 10000LL;
#endif
constexpr uint8_t       rtcI2Caddr      = 0x68;
constexpr uint8_t       bme280I2Caddr   = 0x76;
constexpr uint8_t       eepromI2Caddr   = 0x57;
constexpr uint8_t       eepromKilobits  = 32;
constexpr size_t        eepromBytes     = eepromKilobits/8*1024;

/// Auxiliary helpers
namespace aux {

void printByte(uint8_t b) 
{
  static const char table[] = "0123456789ABCDEF";
  Serial.print(table[(b & 0xF0) >> 4]);
  Serial.print(table[ b & 0x0F]);
};

void printArray(const uint8_t* data, uint8_t len)
{
  for (int i = 0; i < len; ++i) {
    printByte(data[i]);
    Serial.print(" ");
  }
  Serial.println("");
}

} // namespace aux

/// Wrapper for I2C bus
class DeviceI2C
{
public:
  using uint = unsigned int;
  enum ErrCode {
    OK,
    TIMEOUT,
    END_TRANSMISSION_FAIL
  };
  /// Constructor
  DeviceI2C(uint8_t address, uint waitMinMs = 0);
  /// Write byte
  ErrCode writeByte(uint8_t) const;
  /// Write two bytes (e.g. addr, value)
  ErrCode writeBytes(uint8_t, uint8_t) const;
  /// Write byte array
  ErrCode writeArray(uint8_t addr, const uint8_t* data, uint8_t len) const;
  /// Read byte array
  ErrCode readArray(uint8_t addr, uint8_t* data, uint8_t len) const;
protected:
  ErrCode awaitData(uint8_t len) const;
  uint8_t m_i2cAddr;
  uint m_waitMin;
  uint m_waitMax;
};



DeviceI2C::DeviceI2C(uint8_t address, uint waitMinMs)
  : m_i2cAddr(address)
  , m_waitMin(waitMinMs)
  , m_waitMax(200)
{  
}


DeviceI2C::ErrCode DeviceI2C::writeByte(uint8_t b) const
{
  Wire.beginTransmission(m_i2cAddr);
  Wire.write(b);
  return (Wire.endTransmission() == 0) ? ErrCode::OK : ErrCode::END_TRANSMISSION_FAIL;
}


DeviceI2C::ErrCode DeviceI2C::writeBytes(uint8_t b0, uint8_t b1) const
{
  Wire.beginTransmission(m_i2cAddr);
  Wire.write(b0);
  Wire.write(b1);
  return (Wire.endTransmission() == 0) ? ErrCode::OK : ErrCode::END_TRANSMISSION_FAIL;
}


DeviceI2C::ErrCode DeviceI2C::writeArray(uint8_t addr, const uint8_t* data, uint8_t len) const
{
  Wire.beginTransmission(m_i2cAddr);
  Wire.write(addr);
  for (int i = 0; i < len; ++i) {
    Wire.write(data[i]);
  }
  return (Wire.endTransmission() == 0) ? ErrCode::OK : ErrCode::END_TRANSMISSION_FAIL;
}


DeviceI2C::ErrCode DeviceI2C::awaitData(uint8_t len) const
{
  constexpr uint8_t retryMs = 5;
  // Wait for data to become available, up to 100ms
  uint counter = m_waitMin;
  delay(m_waitMin);
  while (Wire.available() < len) {
    counter += retryMs;
    delay(retryMs);
    if (counter > m_waitMax)
      return ErrCode::TIMEOUT;
  }
  return ErrCode::OK;
}


DeviceI2C::ErrCode DeviceI2C::readArray(uint8_t addr, uint8_t* data, uint8_t len) const
{
  DeviceI2C::ErrCode errCode;
  errCode = writeByte(addr);
  if (errCode != ErrCode::OK) {
    return errCode;
  }
  Wire.requestFrom(m_i2cAddr, len);
  errCode = awaitData(len);
  if (errCode != ErrCode::OK) {
    return errCode;
  }
  Wire.readBytes(data, len);
  return ErrCode::OK;
}

////////////////////////////////////////////////////////////////////
class Bme280
{
public:  
  /// Constructor
  explicit Bme280(uint8_t i2cAddr = 0x76);
  /// Initialize measurement
  void begin();
  /// Read chip id
  DeviceI2C::ErrCode readChipId(uint8_t& data) const;
  /// Read calibration data (as of 2024-03-19 only for temperature and pressure)
  DeviceI2C::ErrCode readCalibData(uint8_t* data) const;
  /// Read raw temperature value
  DeviceI2C::ErrCode readRawTemperature(uint8_t* data) const;
  /// Read raw pressure value
  DeviceI2C::ErrCode readRawPressure(uint8_t* data) const;
  /// Read raw humidity value (BME only)
  DeviceI2C::ErrCode readRawHumidity(uint8_t* data) const;
  /// Read all at once (8 bytes)
  DeviceI2C::ErrCode readRawData(uint8_t* data) const;
  /// Resets chip
  DeviceI2C::ErrCode reset() const;
  static constexpr uint8_t RAW_TEMPERATURE_LEN = 3;   // 3 bytes (20bits)
  static constexpr uint8_t RAW_HUMIDITY_LEN    = 2;   // 2 bytes (16bits)
  static constexpr uint8_t RAW_PRESSURE_LEN    = 3;   // 3 bytes (20bits)
  static constexpr uint8_t CALIB_DATA_LEN      = 26;  // 26 bytes for temp and pressure cal.
  /// Decode id as a string
  /// \returns pointer to static buffer
  static const char* decodeId(uint8_t); 
private:

  static constexpr uint8_t REG_CHIP_ID    = 0xD0;
  static constexpr uint8_t REG_RESET      = 0xE0;
  static constexpr uint8_t REG_CALIB_DATA = 0x88; // 0x88-0xA1 26 bytes of clibration data

  static constexpr uint8_t REG_STATUS     = 0xF3;
  static constexpr uint8_t REG_CTRL_MEAS  = 0xF4;
  static constexpr uint8_t REG_CONFIG     = 0xF5;

  static constexpr uint8_t REG_PRESS_MSB  = 0xF7; ///< Starting address of pressure data
  static constexpr uint8_t REG_TEMP_MSB   = 0xFA; ///< Starting address of temperature data
  static constexpr uint8_t REG_HUM_MSB    = 0xFD; ///< Starting address of humidity data (BME280 only)

  DeviceI2C m_bus;  ///< Simple bus driver
  uint8_t m_chipId; ///< ChipID 
};


Bme280::Bme280(uint8_t i2cAddr)
  : m_bus(i2cAddr)
  , m_chipId(0)
{
}


void Bme280::begin()
{
  readChipId(m_chipId);
  reset();
  m_bus.writeBytes(REG_CTRL_MEAS, 0x27);
  m_bus.writeBytes(REG_CONFIG,    0x00);

}

DeviceI2C::ErrCode Bme280::reset() const 
{
  auto err = m_bus.writeBytes(REG_RESET, 0xB6);
  // Delay to ensure the reset process completes
  delay(10); 
  return err;
}


DeviceI2C::ErrCode Bme280::readChipId(uint8_t& data) const
{
  return m_bus.readArray(REG_CHIP_ID, &data, 1);
}


DeviceI2C::ErrCode Bme280::readCalibData(uint8_t* data) const
{
  return m_bus.readArray(REG_CALIB_DATA, data, CALIB_DATA_LEN);
}


DeviceI2C::ErrCode Bme280::readRawTemperature(uint8_t* data) const 
{
  return m_bus.readArray(REG_TEMP_MSB, data, RAW_TEMPERATURE_LEN);
}


DeviceI2C::ErrCode Bme280::readRawPressure(uint8_t* data) const 
{
  return m_bus.readArray(REG_PRESS_MSB, data, RAW_PRESSURE_LEN);
}


DeviceI2C::ErrCode Bme280::readRawHumidity(uint8_t* data) const 
{
  return m_bus.readArray(REG_HUM_MSB, data, RAW_HUMIDITY_LEN);
}


DeviceI2C::ErrCode Bme280::readRawData(uint8_t* data) const 
{
  if (m_chipId == 0x60) {
    return m_bus.readArray(REG_PRESS_MSB, data, RAW_PRESSURE_LEN + RAW_TEMPERATURE_LEN + RAW_HUMIDITY_LEN);
  } else {
    data[6] = data[7] = 0x00;
    return m_bus.readArray(REG_PRESS_MSB, data, RAW_PRESSURE_LEN + RAW_TEMPERATURE_LEN);
  }
}


const char* Bme280::decodeId(uint8_t id)
{
  static const char *table[] = { "UNKNOWN", "BMP180", "BMP280", "BME280" };
  switch(id) {
    case 0x55: return table[1];
    case 0x58: return table[2];
    case 0x60: return table[3];
  }
  return table[0];

}

////////////////////////////////////////////////////////////////////////////////////////////////////////

class Rtc3231
{
public:
  explicit Rtc3231(uint8_t i2cAddr = 0x68);
  /// \brief  Get datetime of DS3231 RTC, assumes UTC (same date format as GPX file)
  /// \return Pointer to static buffer with time  
  static constexpr size_t RTC_TIME_DATA_LEN = 7;
  static constexpr size_t DECODED_TIME_LEN = 22;
  DeviceI2C::ErrCode readTimeData(uint8_t *buf) const;
  static void formatTimeData(char *str, const uint8_t *buf);
private:
  static constexpr uint8_t REG_TIME_DATA = 0x00;
  static void decodeBcd(char *out, uint8_t);
   DeviceI2C m_bus;
};


Rtc3231::Rtc3231(uint8_t i2cAddr)
  : m_bus(i2cAddr, 0)
{
}


void Rtc3231::decodeBcd(char *out, uint8_t b)
{
  out[0] = '0' + (b >> 4);
  out[1] = '0' + (b & 0x0F);
}


DeviceI2C::ErrCode Rtc3231::readTimeData(uint8_t *buf) const
{
  return m_bus.readArray(REG_TIME_DATA, buf, RTC_TIME_DATA_LEN);
}


void Rtc3231::formatTimeData(char *str, const uint8_t *buf)
{
  //                                 012345678901234567890
  static const char iso8601Time[] = "20YY-MM-DDThh:mm:ssZ\0";
  memcpy(str, iso8601Time, DECODED_TIME_LEN);  
  decodeBcd(str+17, buf[0]);        // 00 seconds
  decodeBcd(str+14, buf[1]);        // 01 minutes  
  decodeBcd(str+11, buf[2] & 0x3F); // 02 hours  (bit 7 is 12/24, bit 6 is AM/PM
                                    // 03 day of week (skip)
  decodeBcd(str+ 8, buf[4]);        // 04 day
  decodeBcd(str+ 5, buf[5]);        // 05 month
  decodeBcd(str+ 2, buf[6]);        // 06 year
}


class Eeprom
{
public:
  Eeprom(uint8_t i2cAddr, size_t capacity)
    : m_i2cAddr(i2cAddr)
    , m_capacity(capacity)
  {    
  }

  void write(uint16_t addr, const void* data, size_t len) const
  {
    Wire.beginTransmission(m_i2cAddr);
    Wire.write((int)(addr >> 8));   // MSB of address
    Wire.write((int)(addr & 0xFF)); // LSB of address
    const uint8_t* ptr = static_cast<const uint8_t*>(data);
    for (size_t i = 0; i < len; i++) {
      Wire.write(ptr[i]); // Write each data byte
    }
    Wire.endTransmission();
    delay(5); // Optional delay for EEPROM write cycle
  }

  void read(uint16_t addr, void* data, size_t len) const
  {
    Wire.beginTransmission(m_i2cAddr);
    Wire.write((int)(addr >> 8));   // MSB of address
    Wire.write((int)(addr & 0xFF)); // LSB of address
    Wire.endTransmission();
    delay(5);
    Wire.requestFrom(m_i2cAddr, len);
    uint8_t* ptr = static_cast<uint8_t*>(data);
    for (size_t i = 0; i < len; i++) {
      ptr[i] = Wire.read(); // Write each data byte
    }
    delay(5); // Optional delay for EEPROM write cycle
  }
private:
  uint8_t m_i2cAddr;  
  size_t  m_capacity;
};

// GLOBALS
Bme280 bme280(bme280I2Caddr);
Rtc3231 rtc(rtcI2Caddr);
Eeprom eeprom(eepromI2Caddr , eepromBytes);

constexpr size_t measurementSize = Rtc3231::RTC_TIME_DATA_LEN + Bme280::RAW_TEMPERATURE_LEN + Bme280::RAW_PRESSURE_LEN;
#if !TEST_RUN
constexpr size_t measurementsMax = eepromBytes / measurementSize;
constexpr float measurementsDays = measurementsMax * frequencyMins / 1440.0f;
#else
constexpr size_t measurementsMax = 6;
#endif
size_t eepromWriteAddr = 0;
size_t measurementsCounter = 0;

void setup()
{
  // put your setup code here, to run once:
  Serial.begin(9600);
  while (!Serial)
    ;
  Wire.begin();    

  Serial.println("=== LOGGER 2024-03-23 ============");    

  // BMP280 code
  bme280.begin();
  Serial.print("BME280 ChipID:\t");
  uint8_t chipId;
  bme280.readChipId(chipId);
  Serial.print(chipId, HEX);
  Serial.print(" -> ");
  Serial.println(Bme280::decodeId(chipId));

  Serial.print("BME280 Calibration Data (");
  Serial.print(Bme280::CALIB_DATA_LEN);
  Serial.print(" bytes):\t");
  uint8_t calibData[Bme280::CALIB_DATA_LEN];
  bme280.readCalibData(calibData);
  aux::printArray(calibData, Bme280::CALIB_DATA_LEN);  

  // RTC code  
  uint8_t rtcData[Rtc3231::RTC_TIME_DATA_LEN];
  rtc.readTimeData(rtcData);
  char timeStr[Rtc3231::DECODED_TIME_LEN];
  rtc.formatTimeData(timeStr, rtcData);
  Serial.print("RTC Time:\t");
  Serial.println(timeStr);

  // eeprom code
#if !TEST_RUN
  Serial.println();
  Serial.println("!!! Measurement starts in 30s");
  Serial.println("!!! Start of EEPROM will be overwritten");
  delay(30000);
#else
  // initialize digital pin LED_BUILTIN as an output.
  pinMode(LED_BUILTIN, OUTPUT);
#endif
}


void loop()
{
  // put your main code here, to run repeatedly:
  uint8_t measurement[Rtc3231::RTC_TIME_DATA_LEN + Bme280::RAW_TEMPERATURE_LEN + Bme280::RAW_PRESSURE_LEN];
  if (measurementsCounter < measurementsMax) {
    rtc.readTimeData(measurement);
    bme280.readRawTemperature(measurement + Rtc3231::RTC_TIME_DATA_LEN);
    bme280.readRawPressure(measurement + Rtc3231::RTC_TIME_DATA_LEN + Bme280::RAW_TEMPERATURE_LEN);
    const size_t addr = measurementsCounter * measurementSize;
    eeprom.write(addr, measurement, measurementSize);
    measurementsCounter++;
#if TEST_RUN    
    digitalWrite(LED_BUILTIN, HIGH);  // turn the LED on (HIGH is the voltage level)
    delay(250);                       // wait for a second
    digitalWrite(LED_BUILTIN, LOW);   // turn the LED off by making the voltage LOW
#endif
  } else {
#if TEST_RUN    
    digitalWrite(LED_BUILTIN, HIGH);  // turn the LED on (HIGH is the voltage level)
#endif
  }

  delay(frequencyMs);
}

Breakdown

This code demonstrates a complete example of a simple data logger using an Arduino board, a BME280 sensor, a DS3231 RTC module, and an external EEPROM chip. Here’s a breakdown of the key components:

• DeviceI2C Class: This class acts as a wrapper for I2C communication, providing methods for writing and reading bytes and arrays to and from I2C devices.
• Bme280 Class: This class encapsulates the communication with the BME280 sensor over the I2C bus. It provides methods to initialize the sensor, read the chip ID, calibration data, and raw temperature, pressure, and humidity values.
• Rtc3231 Class: This class handles the communication with the DS3231 RTC module over the I2C bus. It provides a method to read the current time and date in a formatted string.

• Eeprom Class: This class manages the read and write operations to the external EEPROM chip using the I2C bus.

• Setup Function: In the setup function, the code initializes the serial communication, Wire library (for I2C), BME280 sensor, and RTC module. It also prints out the chip ID, calibration data, and current time for diagnostic purposes.
• Loop Function: The loop function is responsible for periodically reading the temperature, pressure, and time data, and writing it to the EEPROM. It first checks if the maximum number of measurements has been reached, and if not, it reads the sensor data and current time, and writes the combined measurement to the EEPROM. The frequency of measurements is determined by the frequencyMs constant.
• Measurement Data Structure: The measurement data is stored in the EEPROM as a contiguous block of bytes. Each measurement consists of the 13 bytes in total:
  • RTC time data (7 bytes),
  • raw temperature data (3 bytes),
  • raw pressure data (3 bytes),

• EEPROM Storage: The code calculates the maximum number of measurements that can be stored in the EEPROM based on its capacity and the size of each measurement. It then writes the measurements sequentially to the EEPROM, starting from address 0.

• TEST_RUN macro: switches between a normal run mode and a test run mode.

  • When TEST_RUN = 0 (the default), the code performs the following:
    • It calculates the maximum number of measurements that can be stored in the EEPROM based on its capacity and the size of each measurement.
    • It waits for 30 seconds before starting the measurement process, providing a warning that the start of the EEPROM will be overwritten.
    • It records measurements at the frequency specified by the frequencyMins constant (10 minutes by default) until the EEPROM is full.

  • When TEST_RUN=1, the code enters a test run mode with the following changes:

    • The maximum number of measurements is fixed at 6, regardless of the EEPROM capacity.
    • The measurement frequency is set to 10 seconds instead of 10 minutes.
    • The built-in LED on the Arduino board blinks after each measurement, providing visual feedback during the test run.

By modifying the constants and adapting the code to your specific hardware configuration, you can tailor this data logger to suit your needs. Additionally, you can expand the code to include functionality for reading the stored data from the EEPROM and processing it further, such as converting the raw sensor values to human-readable units or transferring the data to a computer for analysis.

Please note that the provided code assumes you have the necessary hardware components (Arduino board, BME280 sensor, DS3231 RTC module, and external EEPROM chip) connected correctly.

Data logger

It’s weekend and weather forecast is “interesting” with temperature drop from 20C (first time this year) to nearly zero on Monday morning. At the time of writing this article, there was already strong, brief rainshower with strong wind gusts and METAR data (source: flightradar24) are showing pressure drop from 1008 to 1003hPa in few hours.

I put data logger to balcony.

LKTB 231330Z 29022G32KT CAVOK 13/04 Q1003 NOSIG
LKTB 231300Z 29021KT CAVOK 14/04 Q1003 NOSIG
LKTB 231230Z 30021G33KT 9999 -SHRA VCTS FEW009CB SCT042 14/07 Q1004 BECMG NSW BKN035
LKTB 231200Z 22015G28KT 9999 -SHRA SCT040 18/07 Q1002 NOSIG
LKTB 231130Z 20014G24KT 160V230 9999 SCT043 20/06 Q1002 NOSIG
LKTB 231100Z 18011KT 9999 FEW043 19/06 Q1003 NOSIG
LKTB 231030Z 19011KT CAVOK 18/07 Q1003 NOSIG
LKTB 231000Z 19010KT CAVOK 17/07 Q1004 NOSIG
LKTB 230930Z 17009KT CAVOK 16/07 Q1005 NOSIG
LKTB 230900Z 18009KT CAVOK 15/07 Q1006 NOSIG
LKTB 230830Z 16006KT 110V190 CAVOK 14/07 Q1006 NOSIG
LKTB 230800Z 12003KT 090V150 CAVOK 12/06 Q1007 NOSIG
LKTB 230730Z 09004KT CAVOK 11/06 Q1007 NOSIG
LKTB 230700Z 08004KT CAVOK 09/05 Q1008 NOSIG
LKTB 230630Z VRB02KT CAVOK 09/04 Q1008 NOSIG
LKTB 230600Z 03004KT CAVOK 10/04 Q1008 NOSIG

Significant events:

• Saturday Noon: warm, strong wind gusts, short rain, temperature drops
• Saturday ~4PM: rain, temperature drops
• Sunday morning: pretty cold
• Sunday ~1PM: rain with snow and hails
• Sunday evening: rain showers

Data analysis

First we need to realiably download data

Even this part can go wrong for many reasons. cat, minicom on Linux and PuTTY on Windows did not work as expected and I either got less or more than 4096 bytes. Roger Meier’s CoolTerm on Windows worked.

CoolTerm application

What worked for me on Linux was this (ignores control character, terminates when there are no data to read for five seconds), but single cat command may work or fail.

stty -F /dev/ttyUSB0 9600 -xcase -icanon min 0 time 50 && cat /dev/ttyUSB0 > dump.bin

Second we need to decode data

I should have carefully examined test run data in the first place.

I noticed weird FF FF FF FF FF FF sequences in binary dump and that some records (including the very first one) look weird and BCD date is certainly not valid. So I asked AI to write Python script that decodes data and fixed errors, but something remained. I wanted to see if at least something is useful.

def decode_bcd(data):
  """Decodes a single byte of BCD data into a decimal digit."""
  return ((data & 0xF0) >> 4) * 10 + (data & 0x0F)
  
def read_file(filename):
    """
    Reads the binary data file and processes each 13-byte chunk.
    """
    with open(filename, 'rb') as f:
        data = f.read()

    for i in range(0, len(data), 13):
        chunk = data[i:i+13]
        if len(chunk) == 13:
            seconds = decode_bcd(chunk[0])
            minutes = decode_bcd(chunk[1])
            hours   = decode_bcd(chunk[2])
            day     = decode_bcd(chunk[4])
            month   = decode_bcd(chunk[5])
            year    = decode_bcd(chunk[6]) + 2000

            # Read remaining 2x3 bytes
            byte1 = chunk[7:10]
            byte2 = chunk[10:13]

            # Print the data
            print(f"{year}-{month:02d}-{day:02d} {hours:02d}:{minutes:02d}:{seconds:02d} 0x{byte1.hex().upper()} 0x{byte2.hex().upper()}")

if __name__ == '__main__':
    filename = r"c:\Users\pavel\Documents\CoolTerm Capture (Untitled_0) 2024-03-25 15-42-48-097.txt"   # Replace with your file name
    read_file(filename)

Here’s the result:

2000-57-74 142:82:24 0x7E3700 0x6EAB00
2024-03-23 11:35:38 0x7CB200 0x6E4C00
2000-03-23 11:45:38 0xFFFFFF 0xFFFFFF
2024-03-23 11:55:38 0x7C8300 0x6E1D00
2024-03-23 12:05:38 0x7B5400 0x6DB903
2000-98-73 132:80:24 0x7B2500 0x6DA700
2024-03-23 12:25:37 0x7B0A00 0x6DAE00
2000-93-23 12:35:37 0xFFFFFF 0xFFFFFF
2024-03-23 12:45:37 0x7AD600 0x6D9E00
2024-03-23 12:55:37 0x7A8A00 0x6D2303
2000-60-73 141:79:24 0x7A8B00 0x6D8800
2024-03-23 13:15:37 0x7A5F00 0x6D8000
2000-85-73 13:25:37 0xFFFFFF 0xFFFFFF
⋮
2024-03-25 13:14:49 0x776D00 0x6C6400
2165-165-165 13:24:49 0xFFFFFF 0xFFFFFF
2024-03-25 13:34:49 0x768300 0x6C1500
2165-165-165 165:165:165 0xFFFFFF 0xFFFFFF
2165-165-165 165:165:165 0xFFFFFF 0xFFFFFF
⋮

Well. Some measurements seem valid. Last ones are not, cause I disconnect logger just before log was full, which is ok.

Debugging

Later I tried to modify Arduino code by setting more test measurements with shorter interval and added some debugging output:

#if TEST_RUN
    digitalWrite(LED_BUILTIN, HIGH);  // turn the LED on (HIGH is the voltage level)
    delay(250);                       // wait for a second
    Serial.print(addr, HEX);
    Serial.print(" (");
    Serial.print(addr, DEC);
    Serial.print("):\t");
    aux::printArray(measurement, measurementSize);
    digitalWrite(LED_BUILTIN, LOW);   // turn the LED off by making the voltage LOW
#endif

And got this output on terminal:

BME280 ChipID:  58 -> BMP280
BME280 Calibration Data (26 bytes):     36 6C 05 68 18 FC A1 8D 93 D6 D0 0B C3 06 3B 01 F9 FF 8C 3C F8 C6 70 17 00 00 
RTC Time:  2024-03-25T20:25:21Z
0 (0):     21 25 20 01 25 03 24 7F 4E 00 6E 37 00 
D (13):    24 25 20 01 25 03 24 7F 4E 00 6E 39 00 
1A (26):   28 25 20 01 25 03 24 7F 4F 00 6E 38 00 
27 (39):   31 25 20 01 25 03 24 7F 4F 00 6E 39 00 
34 (52):   34 25 20 01 25 03 24 7F 52 00 6E 38 00 
41 (65):   37 25 20 01 25 03 24 7F 52 00 6E 38 00 
4E (78):   41 25 20 01 25 03 24 7F 51 00 6E 37 00 
5B (91):   44 25 20 01 25 03 24 7F 51 00 6E 3A 00 
68 (104):  47 25 20 01 25 03 24 7F 52 00 6E 39 00 
75 (117):  50 25 20 01 25 03 24 7F 52 00 6E 39 00 

This suggest that good data were read from sensor, whereas EEPROM content is the following

# Print 13 bytes per row by 1 byte, format as hex, add new line, decode 130 bytes
hexdump -ve '13/1 " %.2x" "\n"' -n 130 dump-new.bin

 24 7f 4f 00 6e 38 00 7f 4e 00 6e 37 00
 24 25 20 01 25 03 24 7f 4e 00 6e 39 00
 28 25 20 01 25 03 00 ff ff ff ff ff ff
 31 25 20 01 25 03 24 7f 4f 00 6e 39 00
 34 25 20 01 25 03 24 7f 52 00 6e 38 03
 24 7f 51 00 6e 3a 00 7f 52 00 6e 38 00
 41 25 20 01 25 03 24 7f 51 00 6e 37 00
 44 25 20 01 25 39 00 ff ff ff ff ff ff
 47 25 20 01 25 03 24 7f 52 00 6e 39 00
 50 25 20 01 25 03 24 7f 52 00 6e 23 03

This suggest that some bad data were written to EEPROM. What is mystery here is why bad data block correspond (almost) exactly with sensor data. Well. Kind of. After some testing I concluded that EEPROM on my RTC module has some fault and certain bytes are stuck or causing problems. It even appeared in test run data:

Then I saw there seem to be some regular pattern. In the end, troubleshooting is described in EEPROM article because it’s more relavant to it.

When I tried next run, data were corrupted again, but then I realized that program that reads EEPROM put one extra byte to the beginning and file had 4097 bytes. Finally expected result with 315 entries times 13 bytes, 4095 bytes total, expected to cover 2 days, 4 hours, 20 minutes:

2024-03-28 21:13:43 0x7A6700 0x6DAE00
2024-03-28 21:23:43 0x770900 0x6CCC00

2024-03-31 01:23:22 0x799F00 0x6D7D00
2024-03-31 01:33:22 0x797900 0x6D6D00

Here it seems that sleep function is not very accurate and even with some small extra delays in the code time drifts other way then expected. Sadly I downloaded data one week after acquisition and i do not have weather record for reference. All I remember is that it was warm, strong south wind and smog situation due to dust from Sahara desert limiting visibility to maybe 3km. Not something I remember. Actually, data are available on Weather Underground. Of coarse my data are from balcony where night temperatures are usually much higher and day temperatures are affected by sun.

Things learned

This article demonstrated the basics of reading and writing to an external EEPROM chip with an Arduino board. However, implementing this functionality in a real-world data logger application unveiled some additional challenges:

• While reading and writing ASCII data over the serial port is straightforward, working with binary data proved more complicated due to control character handling and line ending conversions across different systems.
• The initial EEPROM write examples used small 2-byte data chunks, which never crossed EEPROM page boundaries. But when writing larger data blocks for the logger, ignoring page boundaries led to corrupt data being written.

These issues highlight that while isolated examples may work as expected, integrating that code into a complete application can surface new problems to solve.

Much like the classic joke:

• A QA engineer walks into a bar and orders 1 beer.
• A QA engineer walks into a bar and orders 999,999,999,999 beers.
• A QA engineer walks into a bar and orders 0 beers.
• A QA engineer walks into a bar and orders -1 beers.
• A QA engineer walks into a bar and orders “asdfjkl;” beers.
• A QA engineer walks into a bar and orders a lizard.
• The first real customer walks into the bar and asks where the bathroom is. The bartender freezes up, becomes unresponsive, and after a few seconds, bursts into flames.

Result

• EEPROM article and code was updated based on this experience.
• Proposed improvement is to detected if some pin is high or low and switch between eeprom dump mode and logging. It’s annoying to upload different program within 30s, especially when it’s needed to select different Arduino model.
• Python script was updated to decode log. May appear on github.

TODO: in the future, upload it to github

Final words

Happy(ier) logging!