cchen/GP2040-CE
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Protobuf Config (#263)

Browse Source 
* Build makefsdata for Linux w/ MUSL

* Restored deflate support to makefsdata

* Added nanopb library

* Initial version of Proto config

* Added AnimationOptions_Proto

* Storage::toJSON() WIP

* Pretty JSON output

* Use std::string instead of std::ostringstream
This approach reduces the binary size by ~200kb

* Simplify ToJSON() and implented repeated support

* Default initialization

* Fixed issue in nanopb_generator.py

nanopb would not generate code section with enum type defines if the
source file does not contain any enums itself.
However enums can be included from other source files.

* Moved Config functions to separate file
Implemented toJson() and fromJSON()

* Restored StorageManager to its pristine state

* Config loading and saving

* Make FlashPROM cache public for the time being

* Implemented api/getConfig and api/setConfig endpoints

* Moved most Gamepad enums to enums.proto

* Use DEFAULT_INPUT_MODE and DEFAULT_DPAD_MODE

* Renamed generic/short enum values

* Moved bulk of config structs to the ConfigLegacy namespace
No more loading / saving of ConfigLegacy structs

* Moved GamepadOptions and GamepadHotkeyEntry to ConfigLegacy

* FlashPROM::start() is no longer needed

* Added more settings to config.proto

* Support for the latest config changes from main

* Store config struct on the heap to avoid stack overflow

* Increase LWIP_HTTPD_POST_MAX_PAYLOAD_LEN to 8kb
We need more to fit the whole config

* Further stack optimizations
Only save config if content has changed

* Removed template to reduce code bloat

* Update config.boardVersion and save after  ConfigUtils::load()
This makes sure that any migrations performed are preserved in EEPROM

* Config instance in StorageManager

* Renamed getAddonOptions() -> getLegacyAddonOptions()
Renamed setAddonOptions() -> setLegacyAddonOptions()

* Added getters for sub-configs

* Migrate PinMappings to Protobuf

* Fixed wrong name caused by overeager renaming

* Migrated reverse addon to Protobuf storage

* Mirrored latest version of AnimationOptions in Protobuf

* Fixed compatibility with arm-none-eabi-gcc 11.3.1

* Fixed crash issue in Release builds

* Fixed board configs

* Added commented out PICO_DEOPTIMIZED_DEBUG to CMakeList.txt
Use this to toggle between optimized / unoptimized builds

* Added support for byte arrays to base64 encoder / decoder

* Store binary data as byte arrays, convert to Base64 in JSON

* Migrated api/getSplashImage and api/setSplashImage

* Change footer magic to invalidate incompatible stored data

* Refactor pin assignment from JSON

* Add bool to int workaround for web-config

* Migrate SOCD Slider addon to ProtoBuf

* Migrate On-Board LED addon to ProtoBuf

* Migrate Analog Input addon to ProtoBuf

* Migrate BOOTSEL button addon to ProtoBuf

* Migrate Buzzer Speaker addon to ProtoBuf

* Migrate Dual Directional Input addon to ProtoBuf

* Migrate Extra Button addon to ProtoBuf

* Migrate I2C Analog ADS1219 addon to ProtoBuf

* Migrate Joystick Selection Slider addon to ProtoBuf

* Migrate Player Number addon to ProtoBuf

* Migrate Turbo addon to ProtoBuf

* Migrate Wii Extension addon to ProtoBuf

* Migrate PS4 Mode addon to ProtoBuf

* Check writeCache when determining whether to save
Due to the delay in FlashPROM we run the risk of delaying the save indefinitely

* We only allow saves from core0. Saves from core1 have to be marshalled to core0.

* Migrate AnimationOptions

* Check serial and signature for size as well

* Removed unncessary cast

* Moved PS4Options to config_legacy.cpp

* Migrate Display addon to ProtoBuf

* Removed old code related to SplashImage

* Removed old code related to AddonOptions

* Removed stale functions from ConfigManager

* Moved OnBoardLedMode to config_legacy.cpp

* Fixed issues caused by the merge

* Added new player LED properties to LEDOptions

* No more usage of ConfigLegacy::GamepadOptions

* Added switchTpShareForDs4 to config.proto

* Conversion from legacy storage to Protobuf

* Return -1 for invalid pins from webconfig

* More conversion from legacy storage to Protobuf

* Validate enums when converting from legacy config

* Remove switchTpShareForDs4 from legacy conversion
It is not part of the 0.7.1 release

* More conversion from legacy storage to Protobuf

* Removed references to ConfigLegacy::BoardOptions

* Fixed compilation of KB2040

* Re-added /api/getConfig function entry
Must have gotten lost in a merge

* Fixed issue where JSLider addon fails to change dpadMode

* Migrated SNES options to Protobuf

* Fixed saving of AnimationOptions

---------

Co-authored-by: deeebug <77402236+deeebug@users.noreply.github.com>
This commit is contained in:
Malte Thiesen
2023-05-25 22:02:08 +02:00
committed by GitHub
parent 3a62d5b317
commit 3987f2037e
633 changed files with 42580 additions and 1861 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
CMakeLists.txt
View File

@@ -12,6 +12,9 @@ if (PICO_SDK_VERSION_STRING VERSION_LESS "1.5.0")
message(FATAL_ERROR "Raspberry Pi Pico SDK version 1.5.0 (or later) required. Your version is ${PICO_SDK_VERSION_STRING}")
endif()
# Uncomment the next line for an unomptimized build for debugging. Use in conjunction with the Debug build type.
# set(PICO_DEOPTIMIZED_DEBUG 1)
project(GP2040-CE LANGUAGES C CXX ASM VERSION 0.7.1)
set(CMAKE_C_STANDARD 11)
set(CMAKE_CXX_STANDARD 17)
@@ -84,6 +87,9 @@ FetchContent_Declare(ArduinoJson
)
FetchContent_MakeAvailable(ArduinoJson)
include(compile_proto.cmake)
compile_proto()
# initialize the Raspberry Pi Pico SDK
pico_sdk_init()
add_subdirectory(lib)
@@ -106,6 +112,8 @@ src/addonmanager.cpp
src/configmanager.cpp
src/storagemanager.cpp
src/system.cpp
src/config_legacy.cpp
src/config_utils.cpp
src/configs/webconfig.cpp
src/addons/analog.cpp
src/addons/board_led.cpp
@@ -127,6 +135,8 @@ src/addons/wiiext.cpp
src/addons/snes_input.cpp
src/gamepad/GamepadDebouncer.cpp
src/gamepad/GamepadDescriptors.cpp
${PROTO_OUTPUT_DIR}/enums.pb.c
${PROTO_OUTPUT_DIR}/config.pb.c
)
set_target_properties(${PROJECT_NAME} PROPERTIES OUTPUT_NAME ${PROJECT_NAME}_${CMAKE_PROJECT_VERSION}_${GP2040_BOARDCONFIG})
@@ -151,6 +161,7 @@ WiiExtension
SNESpad
pico_mbedtls
TinyUSB_Gamepad
nanopb
)
target_include_directories(${PROJECT_NAME} PUBLIC
@@ -159,6 +170,7 @@ headers/addons
headers/configs
headers/gamepad
configs/${GP2040_BOARDCONFIG}
${PROTO_OUTPUT_DIR}
)
target_compile_definitions(${PROJECT_NAME} PUBLIC

52
compile_proto.cmake Normal file
View File

@@ -0,0 +1,52 @@
function (compile_proto)
find_package(Python3 REQUIRED COMPONENTS Interpreter)
set(VENV ${CMAKE_CURRENT_BINARY_DIR}/venv)
set(VENV_FILE ${VENV}/environment.txt)
if(CMAKE_HOST_WIN32)
set(VENV_BIN_DIR ${VENV}/Scripts)
else()
set(VENV_BIN_DIR ${VENV}/bin)
endif()
add_custom_command(
DEPENDS ${CMAKE_SOURCE_DIR}/lib/nanopb/extra/requirements.txt
COMMAND ${Python3_EXECUTABLE} -m venv ${VENV}
COMMAND ${VENV_BIN_DIR}/pip --disable-pip-version-check install -r ${CMAKE_SOURCE_DIR}/lib/nanopb/extra/requirements.txt
COMMAND ${VENV_BIN_DIR}/pip freeze > ${VENV_FILE}
OUTPUT ${VENV_FILE}
COMMENT "Setting up Python Virtual Environment"
)
set(NANOPB_GENERATOR ${CMAKE_SOURCE_DIR}/lib/nanopb/generator/nanopb_generator.py)
set(PROTO_OUTPUT_DIR ${CMAKE_CURRENT_BINARY_DIR}/proto)
set(PROTO_OUTPUT_DIR ${PROTO_OUTPUT_DIR} PARENT_SCOPE)
add_custom_command(
DEPENDS ${VENV_FILE} ${NANOPB_GENERATOR} ${CMAKE_SOURCE_DIR}/proto/enums.proto ${CMAKE_SOURCE_DIR}/lib/nanopb/generator/proto/nanopb.proto
WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}
COMMAND ${CMAKE_COMMAND} -E make_directory ${PROTO_OUTPUT_DIR}
COMMAND ${VENV_BIN_DIR}/python ${NANOPB_GENERATOR}
-q
-D ${PROTO_OUTPUT_DIR}
-I ${CMAKE_SOURCE_DIR}/proto
-I ${CMAKE_SOURCE_DIR}/lib/nanopb/generator/proto
${CMAKE_SOURCE_DIR}/proto/enums.proto
OUTPUT ${PROTO_OUTPUT_DIR}/enums.pb.c ${PROTO_OUTPUT_DIR}/enums.pb.h
COMMENT "Compiling enums.proto"
)
add_custom_command(
DEPENDS ${VENV_FILE} ${NANOPB_GENERATOR} ${CMAKE_SOURCE_DIR}/proto/enums.proto ${CMAKE_SOURCE_DIR}/proto/config.proto ${CMAKE_SOURCE_DIR}/lib/nanopb/generator/proto/nanopb.proto
WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}
COMMAND ${CMAKE_COMMAND} -E make_directory ${PROTO_OUTPUT_DIR}
COMMAND ${VENV_BIN_DIR}/python ${NANOPB_GENERATOR}
-q
-D ${PROTO_OUTPUT_DIR}
-I ${CMAKE_SOURCE_DIR}/proto
-I ${CMAKE_SOURCE_DIR}/lib/nanopb/generator/proto
${CMAKE_SOURCE_DIR}/proto/config.proto
OUTPUT ${PROTO_OUTPUT_DIR}/config.pb.c ${PROTO_OUTPUT_DIR}/config.pb.h
COMMENT "Compiling config.proto"
)
endfunction()

8
configs/BentoBox/BoardConfig.h
View File

@@ -6,7 +6,7 @@
#ifndef PICO_BOARD_CONFIG_H_
#define PICO_BOARD_CONFIG_H_
#include <GamepadEnums.h>
#include "enums.pb.h"
// This is the main pin definition section.
// This will let you specify which GPIO pin each button is assigned too.
@@ -189,8 +189,8 @@
#define BUTTON_LAYOUT BUTTON_LAYOUT_STICKLESS
#define BUTTON_LAYOUT_RIGHT BUTTON_LAYOUT_STICKLESSB
#define SPLASH_MODE NOSPLASH
#define SPLASH_CHOICE MAIN
#define SPLASH_MODE SPLASH_MODE_NONE
#define SPLASH_CHOICE SPLASH_CHOICE_MAIN
#define SPLASH_DURATION 7500 // Duration in milliseconds
// Board LED Add-on Setting
@@ -199,7 +199,7 @@
// on the current mode (config, normal, or no USB data)
// INPUT_TEST - Blinks whenever any input is made
#define BOARD_LED_TYPE BOARD_LED_OFF
#define BOARD_LED_TYPE ON_BOARD_LED_MODE_OFF
// Dual Directional Add-on Options

9
configs/DURAL/BoardConfig.h
View File

@@ -6,7 +6,8 @@
#ifndef DURAL_CONFIG_H_
#define DURAL_CONFIG_H_
#include <GamepadEnums.h>
#include "enums.pb.h"
// This is the main pin definition section.
// This will let you specify which GPIO pin each button is assigned too.
// You can set any of the main pins as `-1` to disable it.
@@ -189,8 +190,8 @@
#define BUTTON_LAYOUT BUTTON_LAYOUT_STICKLESS
#define BUTTON_LAYOUT_RIGHT BUTTON_LAYOUT_STICKLESSB
#define SPLASH_MODE NOSPLASH
#define SPLASH_CHOICE MAIN
#define SPLASH_MODE SPLASH_MODE_NONE
#define SPLASH_CHOICE SPLASH_CHOICE_MAIN
#define SPLASH_DURATION 7500 // Duration in milliseconds
// Board LED Add-on Setting
@@ -199,7 +200,7 @@
// on the current mode (config, normal, or no USB data)
// INPUT_TEST - Blinks whenever any input is made
#define BOARD_LED_TYPE BOARD_LED_OFF
#define BOARD_LED_TYPE ON_BOARD_LED_MODE_OFF
// Dual Directional Add-on Options

8
configs/FightboardV3/BoardConfig.h
View File

@@ -6,7 +6,7 @@
#ifndef PICO_BOARD_CONFIG_H_
#define PICO_BOARD_CONFIG_H_
#include <GamepadEnums.h>
#include "enums.pb.h"
// This is the main pin definition section.
@@ -198,8 +198,8 @@
#define BUTTON_LAYOUT BUTTON_LAYOUT_FIGHTBOARD_STICK
#define BUTTON_LAYOUT_RIGHT BUTTON_LAYOUT_FIGHTBOARD
#define SPLASH_MODE NOSPLASH
#define SPLASH_CHOICE MAIN
#define SPLASH_MODE SPLASH_MODE_NONE
#define SPLASH_CHOICE SPLASH_CHOICE_MAIN
#define SPLASH_DURATION 7500 // Duration in milliseconds
// Default theme
@@ -214,7 +214,7 @@
// on the current mode (config, normal, or no USB data)
// INPUT_TEST - Blinks whenever any input is made
#define BOARD_LED_TYPE BOARD_LED_OFF
#define BOARD_LED_TYPE ON_BOARD_LED_MODE_OFF
// Dual Directional Add-on Options
#define PIN_DUAL_DIRECTIONAL_UP -1

8
configs/FightboardV3Mirrored/BoardConfig.h
View File

@@ -6,7 +6,7 @@
#ifndef PICO_BOARD_CONFIG_H_
#define PICO_BOARD_CONFIG_H_
#include <GamepadEnums.h>
#include "enums.pb.h"
// This is the main pin definition section.
@@ -198,8 +198,8 @@
#define BUTTON_LAYOUT BUTTON_LAYOUT_FIGHTBOARD_MIRRORED
#define BUTTON_LAYOUT_RIGHT BUTTON_LAYOUT_FIGHTBOARD_STICK_MIRRORED
#define SPLASH_MODE NOSPLASH
#define SPLASH_CHOICE MAIN
#define SPLASH_MODE SPLASH_MODE_NONE
#define SPLASH_CHOICE SPLASH_CHOICE_MAIN
#define SPLASH_DURATION 7500 // Duration in milliseconds
// Default theme
@@ -214,7 +214,7 @@
// on the current mode (config, normal, or no USB data)
// INPUT_TEST - Blinks whenever any input is made
#define BOARD_LED_TYPE BOARD_LED_OFF
#define BOARD_LED_TYPE ON_BOARD_LED_MODE_OFF
// Dual Directional Add-on Options
#define PIN_DUAL_DIRECTIONAL_UP -1

9
configs/FlatboxRev4/BoardConfig.h
View File

@@ -6,7 +6,8 @@
#ifndef FLATBOX_REV4_CONFIG_H_
#define FLATBOX_REV4_CONFIG_H_
#include <GamepadEnums.h>
#include "enums.pb.h"
// Mapping between Flatbox Rev4 switch number (as silkscreened) and GPIO pin
#define FLATBOX_SW1_PIN 6
@@ -158,8 +159,8 @@
// 4 - `NOSPLASH` - This will not display a splash screen on boot
// Special note - All of the splash screen images can be changed via `include/bitmaps.h`
#define SPLASH_MODE NOSPLASH
#define SPLASH_CHOICE MAIN
#define SPLASH_MODE SPLASH_MODE_NONE
#define SPLASH_CHOICE SPLASH_CHOICE_MAIN
#define SPLASH_DURATION 7500 // Duration in milliseconds
@@ -200,7 +201,7 @@
// on the current mode (config, normal, or no USB data)
// INPUT_TEST - Blinks whenever any input is made
#define BOARD_LED_TYPE BOARD_LED_OFF
#define BOARD_LED_TYPE ON_BOARD_LED_MODE_OFF
// Dual Directional Add-on Options

9
configs/FlatboxRev5/BoardConfig.h
View File

@@ -6,7 +6,8 @@
#ifndef FLATBOX_REV5_CONFIG_H_
#define FLATBOX_REV5_CONFIG_H_
#include <GamepadEnums.h>
#include "enums.pb.h"
// Mapping between Flatbox rev5 switch number (as silkscreened) and GPIO pin
#define FLATBOX_SW1_PIN 14
@@ -158,8 +159,8 @@
// 4 - `NOSPLASH` - This will not display a splash screen on boot
// Special note - All of the splash screen images can be changed via `include/bitmaps.h`
#define SPLASH_MODE NOSPLASH
#define SPLASH_CHOICE MAIN
#define SPLASH_MODE SPLASH_MODE_NONE
#define SPLASH_CHOICE SPLASH_CHOICE_MAIN
#define SPLASH_DURATION 7500 // Duration in milliseconds
@@ -200,7 +201,7 @@
// on the current mode (config, normal, or no USB data)
// INPUT_TEST - Blinks whenever any input is made
#define BOARD_LED_TYPE BOARD_LED_OFF
#define BOARD_LED_TYPE ON_BOARD_LED_MODE_OFF
// Dual Directional Add-on Options

6
configs/KB2040/BoardConfig.h
View File

@@ -202,8 +202,8 @@
#define JSLIDER_ENABLED 1
#define BUTTON_LAYOUT BUTTON_LAYOUT_STICK
#define BUTTON_LAYOUT_RIGHT BUTTON_LAYOUT_VEWLIX
#define SPLASH_MODE NOSPLASH
#define SPLASH_CHOICE MAIN
#define SPLASH_MODE SPLASH_MODE_NONE
#define SPLASH_CHOICE SPLASH_CHOICE_MAIN
#define SPLASH_DURATION 7500 // Duration in milliseconds
// Board LED Add-on Setting
@@ -212,7 +212,7 @@
// on the current mode (config, normal, or no USB data)
// INPUT_TEST - Blinks whenever any input is made
#define BOARD_LED_TYPE BOARD_LED_OFF
#define BOARD_LED_TYPE ON_BOARD_LED_MODE_OFF
// Dual Directional Add-on Options

9
configs/MavercadeKeebfighter/BoardConfig.h
View File

@@ -6,7 +6,8 @@
#ifndef MAVERCADEKEEBFIGHTER_CONFIG_H_
#define MAVERCADEKEEBFIGHTER_CONFIG_H_
#include <GamepadEnums.h>
#include "enums.pb.h"
// Mapping between Mavercade Keebfighter switch number (as silkscreened) and GPIO pin
#define FLATBOX_SW1_PIN 14
@@ -177,8 +178,8 @@
// 4 - `NOSPLASH` - This will not display a splash screen on boot
// Special note - All of the splash screen images can be changed via `include/bitmaps.h`
#define SPLASH_MODE NOSPLASH
#define SPLASH_CHOICE MAIN
#define SPLASH_MODE SPLASH_MODE_NONE
#define SPLASH_CHOICE SPLASH_CHOICE_MAIN
// The default `BUTTON_LAYOUT` is `BUTTON_LAYOUT_STICK` which will show an arcade stick on the left hand side of the display.
@@ -218,7 +219,7 @@
// on the current mode (config, normal, or no USB data)
// INPUT_TEST - Blinks whenever any input is made
#define BOARD_LED_TYPE BOARD_LED_OFF
#define BOARD_LED_TYPE ON_BOARD_LED_MODE_OFF
// Dual Directional Add-on Options

8
configs/Pico/BoardConfig.h
View File

@@ -6,7 +6,7 @@
#ifndef PICO_BOARD_CONFIG_H_
#define PICO_BOARD_CONFIG_H_
#include <GamepadEnums.h>
#include "enums.pb.h"
// This is the main pin definition section.
// This will let you specify which GPIO pin each button is assigned too.
@@ -199,8 +199,8 @@
#define BUTTON_LAYOUT BUTTON_LAYOUT_STICK
#define BUTTON_LAYOUT_RIGHT BUTTON_LAYOUT_VEWLIX
#define SPLASH_MODE NOSPLASH
#define SPLASH_CHOICE MAIN
#define SPLASH_MODE SPLASH_MODE_NONE
#define SPLASH_CHOICE SPLASH_CHOICE_MAIN
#define SPLASH_DURATION 7500 // Duration in milliseconds
// Board LED Add-on Setting
@@ -209,7 +209,7 @@
// on the current mode (config, normal, or no USB data)
// INPUT_TEST - Blinks whenever any input is made
#define BOARD_LED_TYPE BOARD_LED_OFF
#define BOARD_LED_TYPE ON_BOARD_LED_MODE_OFF
// Dual Directional Add-on Options

9
configs/PicoAnn/BoardConfig.h
View File

@@ -6,7 +6,8 @@
#ifndef PICOANN_CONFIG_H_
#define PICOANN_CONFIG_H_
#include <GamepadEnums.h>
#include "enums.pb.h"
// This is the main pin definition section.
// This will let you specify which GPIO pin each button is assigned too.
// You can set any of the main pins as `-1` to disable it.
@@ -189,8 +190,8 @@
#define BUTTON_LAYOUT BUTTON_LAYOUT_STICK
#define BUTTON_LAYOUT_RIGHT BUTTON_LAYOUT_VEWLIX
#define SPLASH_MODE NOSPLASH
#define SPLASH_CHOICE MAIN
#define SPLASH_MODE SPLASH_MODE_NONE
#define SPLASH_CHOICE SPLASH_CHOICE_MAIN
#define SPLASH_DURATION 7500 // Duration in milliseconds
// Board LED Add-on Setting
@@ -199,7 +200,7 @@
// on the current mode (config, normal, or no USB data)
// INPUT_TEST - Blinks whenever any input is made
#define BOARD_LED_TYPE BOARD_LED_OFF
#define BOARD_LED_TYPE ON_BOARD_LED_MODE_OFF
// Dual Directional Add-on Options

9
configs/PicoFightingBoard/BoardConfig.h
View File

@@ -6,7 +6,8 @@
#ifndef PICO_BOARD_CONFIG_H_
#define PICO_BOARD_CONFIG_H_
#include <GamepadEnums.h>
#include "enums.pb.h"
// This is the main pin definition section.
// This will let you specify which GPIO pin each button is assigned too.
// You can set any of the main pins as `-1` to disable it.
@@ -202,8 +203,8 @@
#define BUTTON_LAYOUT BUTTON_LAYOUT_STICKLESS
#define BUTTON_LAYOUT_RIGHT BUTTON_LAYOUT_STICKLESSB
#define SPLASH_MODE NOSPLASH
#define SPLASH_CHOICE MAIN
#define SPLASH_MODE SPLASH_MODE_NONE
#define SPLASH_CHOICE SPLASH_CHOICE_MAIN
#define SPLASH_DURATION 7500 // Duration in milliseconds
// Board LED Add-on Setting
@@ -212,7 +213,7 @@
// on the current mode (config, normal, or no USB data)
// INPUT_TEST - Blinks whenever any input is made
#define BOARD_LED_TYPE BOARD_LED_OFF
#define BOARD_LED_TYPE ON_BOARD_LED_MODE_OFF
// Dual Directional Add-on Options

9
configs/RP2040AdvancedBreakoutBoard/BoardConfig.h
View File

@@ -6,7 +6,8 @@
#ifndef PICO_BOARD_CONFIG_H_
#define PICO_BOARD_CONFIG_H_
#include <GamepadEnums.h>
#include "enums.pb.h"
// This is the main pin definition section.
// This will let you specify which GPIO pin each button is assigned too.
// You can set any of the main pins as `-1` to disable it.
@@ -189,8 +190,8 @@
#define BUTTON_LAYOUT BUTTON_LAYOUT_STICK
#define BUTTON_LAYOUT_RIGHT BUTTON_LAYOUT_VEWLIX
#define SPLASH_MODE NOSPLASH
#define SPLASH_CHOICE MAIN
#define SPLASH_MODE SPLASH_MODE_NONE
#define SPLASH_CHOICE SPLASH_CHOICE_MAIN
#define SPLASH_DURATION 7500 // Duration in milliseconds
// Board LED Add-on Setting
@@ -199,7 +200,7 @@
// on the current mode (config, normal, or no USB data)
// INPUT_TEST - Blinks whenever any input is made
#define BOARD_LED_TYPE MODE_INDICATOR
#define BOARD_LED_TYPE ON_BOARD_LED_MODE_MODE_INDICATOR
#define BOARD_LED_PIN 25
// Dual Directional Add-on Options

9
configs/ReflexEncoder/BoardConfig.h
View File

@@ -6,7 +6,8 @@
#ifndef PICO_BOARD_CONFIG_H_
#define PICO_BOARD_CONFIG_H_
#include <GamepadEnums.h>
#include "enums.pb.h"
// This is the main pin definition section.
// This will let you specify which GPIO pin each button is assigned too.
// You can set any of the main pins as `-1` to disable it.
@@ -202,8 +203,8 @@
#define BUTTON_LAYOUT BUTTON_LAYOUT_STICKLESS
#define BUTTON_LAYOUT_RIGHT BUTTON_LAYOUT_STICKLESSB
#define SPLASH_MODE NOSPLASH
#define SPLASH_CHOICE MAIN
#define SPLASH_MODE SPLASH_MODE_NONE
#define SPLASH_CHOICE SPLASH_CHOICE_MAIN
#define SPLASH_DURATION 7500 // Duration in milliseconds
// Board LED Add-on Setting
@@ -212,7 +213,7 @@
// on the current mode (config, normal, or no USB data)
// INPUT_TEST - Blinks whenever any input is made
#define BOARD_LED_TYPE BOARD_LED_OFF
#define BOARD_LED_TYPE ON_BOARD_LED_MODE_OFF
// Dual Directional Add-on Options

8
configs/SparkFunProMicro/BoardConfig.h
View File

@@ -1,7 +1,7 @@
#ifndef PICO_BOARD_CONFIG_H_
#define PICO_BOARD_CONFIG_H_
#include <GamepadEnums.h>
#include "enums.pb.h"
// This is the main pin definition section.
// This will let you specify which GPIO pin each button is assigned too.
@@ -180,8 +180,8 @@
#define BUTTON_LAYOUT BUTTON_LAYOUT_STICK
#define BUTTON_LAYOUT_RIGHT BUTTON_LAYOUT_VEWLIX
#define SPLASH_MODE NOSPLASH
#define SPLASH_CHOICE MAIN
#define SPLASH_MODE SPLASH_MODE_NONE
#define SPLASH_CHOICE SPLASH_CHOICE_MAIN
#define SPLASH_DURATION 7500 // Duration in milliseconds
// Board LED Add-on Setting
@@ -190,7 +190,7 @@
// on the current mode (config, normal, or no USB data)
// INPUT_TEST - Blinks whenever any input is made
#define BOARD_LED_TYPE BOARD_LED_OFF
#define BOARD_LED_TYPE ON_BOARD_LED_MODE_OFF
// Dual Directional Add-on Options

9
configs/WaveshareZero/BoardConfig.h
View File

@@ -6,7 +6,8 @@
#ifndef PICO_BOARD_CONFIG_H_
#define PICO_BOARD_CONFIG_H_
#include <GamepadEnums.h>
#include "enums.pb.h"
// This is the main pin definition section.
// This will let you specify which GPIO pin each button is assigned too.
// You can set any of the main pins as `-1` to disable it.
@@ -189,8 +190,8 @@
#define BUTTON_LAYOUT BUTTON_LAYOUT_STICK
#define BUTTON_LAYOUT_RIGHT BUTTON_LAYOUT_VEWLIX
#define SPLASH_MODE NOSPLASH
#define SPLASH_CHOICE MAIN
#define SPLASH_MODE SPLASH_MODE_NONE
#define SPLASH_CHOICE SPLASH_CHOICE_MAIN
#define SPLASH_DURATION 7500 // Duration in milliseconds
// Board LED Add-on Setting
@@ -199,7 +200,7 @@
// on the current mode (config, normal, or no USB data)
// INPUT_TEST - Blinks whenever any input is made
#define BOARD_LED_TYPE BOARD_LED_OFF
#define BOARD_LED_TYPE ON_BOARD_LED_MODE_OFF
// Dual Directional Add-on Options

2
headers/addons/dualdirectional.h
View File

@@ -59,7 +59,7 @@ private:
uint8_t SOCDCombine(SOCDMode, uint8_t);
uint8_t SOCDGamepadClean(uint8_t, bool isLastWin);
void OverrideGamepad(Gamepad *, DpadMode, uint8_t);
const SOCDMode getSOCDMode(GamepadOptions&);
const SOCDMode getSOCDMode(const GamepadOptions&);
uint8_t dDebState; // Debounce State (stored)
uint8_t dualState; // Dual Directional State
DpadDirection lastGPUD; // Gamepad Last Up-Down

6
headers/addons/i2cdisplay.h
View File

@@ -108,8 +108,8 @@ private:
void drawBlankB(int startX, int startY, int buttonSize, int buttonPadding);
void drawVLXA(int startX, int startY, int buttonRadius, int buttonPadding);
void drawVLXB(int startX, int startY, int buttonRadius, int buttonPadding);
void drawButtonLayoutLeft(ButtonLayoutCustomOptions options);
void drawButtonLayoutRight(ButtonLayoutCustomOptions options);
void drawButtonLayoutLeft(ButtonLayoutParamsLeft& options);
void drawButtonLayoutRight(ButtonLayoutParamsRight& options);
void drawFightboard(int startX, int startY, int buttonRadius, int buttonPadding);
void drawFightboardMirrored(int startX, int startY, int buttonRadius, int buttonPadding);
void drawFightboardStick(int startX, int startY, int buttonRadius, int buttonPadding);
@@ -118,7 +118,7 @@ private:
bool pressedDown();
bool pressedLeft();
bool pressedRight();
const BoardOptions& getBoardOptions();
const DisplayOptions& getDisplayOptions();
bool isDisplayPowerOff();
void setDisplayPower(uint8_t status);
uint32_t displaySaverTimeout = 0;

4
headers/addons/pleds.h
View File

@@ -14,6 +14,8 @@
#include "gpaddon.h"
#include "helper.h"
#include "enums.pb.h"
// This needs to be moved to storage if we're going to share between modules
extern NeoPico *neopico;
extern AnimationStation as;
@@ -38,7 +40,7 @@ public:
virtual void process();
virtual std::string name() { return PLEDName; }
PlayerLEDAddon() {
type = static_cast<PLEDType>(Storage::getInstance().getLEDOptions().pledType);
type = static_cast<PLEDType>(Storage::getInstance().getLedOptions().pledType);
}
PlayerLEDAddon(PLEDType type) : type(type) {}

14
headers/addons/ps4mode.h
View File

@@ -4,6 +4,8 @@
#include "gpaddon.h"
#include "storagemanager.h"
#include "mbedtls/rsa.h"
#ifndef PS4MODE_ADDON_ENABLED
#define PS4MODE_ADDON_ENABLED 0
#endif
@@ -20,8 +22,16 @@ public:
virtual std::string name() { return PS4ModeName; }
private:
struct mbedtls_rsa_context rsa_context;
uint8_t hashed_nonce[32];
PS4Options ps4Options;
mbedtls_mpi_uint bytesN[64] = {};
mbedtls_mpi_uint bytesE[1] = {};
mbedtls_mpi_uint bytesD[64] = {};
mbedtls_mpi_uint bytesP[32] = {};
mbedtls_mpi_uint bytesQ[32] = {};
mbedtls_mpi_uint bytesDP[32] = {};
mbedtls_mpi_uint bytesDQ[32] = {};
mbedtls_mpi_uint bytesQP[32] = {};
mbedtls_mpi_uint bytesRN[64] = {};
uint8_t hashed_nonce[32] = {};
};
#endif // PS4MODE_H_

10
headers/addons/turbo.h
View File

@@ -3,6 +3,7 @@
#include "gpaddon.h"
#include "storagemanager.h"
#include "enums.pb.h"
#ifndef TURBO_ENABLED
#define TURBO_ENABLED 0
@@ -26,13 +27,8 @@
#define TURBO_SHMUP_MODE 0
#endif
enum ShmupMixMode {
TURBO_PRIORITY = 0,
CHARGE_PRIORITY
};
#ifndef SHMUP_MIX_MODE
#define SHMUP_MIX_MODE TURBO_PRIORITY
#define SHMUP_MIX_MODE SHMUP_MIX_MODE_TURBO_PRIORITY
#endif
#ifndef SHMUP_ALWAYS_ON1
@@ -98,7 +94,7 @@ public:
virtual void process(); // TURBO Setting of buttons (Enable/Disable)
virtual std::string name() { return TurboName; }
private:
void read(const AddonOptions&); // Read TURBO Buttons and Dials
void read(const TurboOptions&); // Read TURBO Buttons and Dials
void debounce(); // TURBO Button Debouncer
void updateTurboShotCount(uint8_t turboShotCount);
bool bDebState; // Debounce TURBO Button State

18
headers/config_utils.h Normal file
View File

@@ -0,0 +1,18 @@
#ifndef CONFIG_UTILS_H
#define CONFIG_UTILS_H
#include "config.pb.h"
#include <string>
namespace ConfigUtils {
void load(Config& config);
bool save(Config& config);
void initUnsetPropertiesWithDefaults(Config& config);
std::string toJSON(const Config& config);
bool fromJSON(Config& config, const char* data, size_t dataLen);
bool fromLegacyStorage(Config& config);
}
#endif

8
headers/configmanager.h
View File

@@ -1,5 +1,5 @@
#ifndef _CONFIG_H_
#define _CONFIG_H_
#ifndef CONFIGMANAGER_H
#define CONFIGMANAGER_H
#include "enums.h"
#include "gpconfig.h"
@@ -16,10 +16,6 @@ public:
void setup(ConfigType);
void loop(); // If anything needs to update in the gpconfig driver
void setGamepadOptions(Gamepad*);
void setBoardOptions(BoardOptions);
void setPreviewBoardOptions(BoardOptions);
void setLedOptions(LEDOptions);
void setSplashImage(const SplashImage&);
private:
ConfigManager() {}
void setupConfig(GPConfig*);

67
headers/configs/base64.h
View File

@@ -30,7 +30,7 @@
class Base64 {
public:
static std::string Encode(const std::string data) {
static std::string Encode(const char* dataPtr, size_t dataLen) {
static constexpr char sEncodingTable[] = {
'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H',
'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P',
@@ -42,27 +42,31 @@ class Base64 {
'4', '5', '6', '7', '8', '9', '+', '/'
};
size_t in_len = data.size();
size_t out_len = 4 * ((in_len + 2) / 3);
std::string ret(out_len, '\0');
size_t i;
char *p = const_cast<char*>(ret.c_str());
size_t out_len = 4 * ((dataLen + 2) / 3);
std::string ret;
ret.resize(out_len);
for (i = 0; i < in_len - 2; i += 3) {
*p++ = sEncodingTable[(data[i] >> 2) & 0x3F];
*p++ = sEncodingTable[((data[i] & 0x3) << 4) | ((int) (data[i + 1] & 0xF0) >> 4)];
*p++ = sEncodingTable[((data[i + 1] & 0xF) << 2) | ((int) (data[i + 2] & 0xC0) >> 6)];
*p++ = sEncodingTable[data[i + 2] & 0x3F];
size_t i = 0;
char *p = ret.data();
if (dataLen >= 2) {
for (; i < dataLen - 2; i += 3) {
*p++ = sEncodingTable[(dataPtr[i] >> 2) & 0x3F];
*p++ = sEncodingTable[((dataPtr[i] & 0x3) << 4) | ((int) (dataPtr[i + 1] & 0xF0) >> 4)];
*p++ = sEncodingTable[((dataPtr[i + 1] & 0xF) << 2) | ((int) (dataPtr[i + 2] & 0xC0) >> 6)];
*p++ = sEncodingTable[dataPtr[i + 2] & 0x3F];
}
}
if (i < in_len) {
*p++ = sEncodingTable[(data[i] >> 2) & 0x3F];
if (i == (in_len - 1)) {
*p++ = sEncodingTable[((data[i] & 0x3) << 4)];
if (i < dataLen) {
*p++ = sEncodingTable[(dataPtr[i] >> 2) & 0x3F];
if (i == (dataLen - 1)) {
*p++ = sEncodingTable[((dataPtr[i] & 0x3) << 4)];
*p++ = '=';
}
else {
*p++ = sEncodingTable[((data[i] & 0x3) << 4) | ((int) (data[i + 1] & 0xF0) >> 4)];
*p++ = sEncodingTable[((data[i + 1] & 0xF) << 2)];
*p++ = sEncodingTable[((dataPtr[i] & 0x3) << 4) | ((int) (dataPtr[i + 1] & 0xF0) >> 4)];
*p++ = sEncodingTable[((dataPtr[i + 1] & 0xF) << 2)];
}
*p++ = '=';
}
@@ -70,7 +74,11 @@ class Base64 {
return ret;
}
static bool Decode(const std::string& input, std::string& out) {
static std::string Encode(const std::string data) {
return Encode(data.data(), data.length());
}
static bool Decode(const char* dataPtr, size_t dataLen, std::string& out) {
static constexpr unsigned char kDecodingTable[] = {
64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64,
64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64,
@@ -90,24 +98,23 @@ class Base64 {
64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64
};
size_t in_len = input.size();
if (in_len % 4 != 0)
if (dataLen % 4 != 0)
{
out.clear();
return false;
}
size_t out_len = in_len / 4 * 3;
if (input[in_len - 1] == '=') out_len--;
if (input[in_len - 2] == '=') out_len--;
size_t out_len = dataLen / 4 * 3;
if (dataLen >= 1 && dataPtr[dataLen - 1] == '=') out_len--;
if (dataLen >= 2 && dataPtr[dataLen - 2] == '=') out_len--;
out.resize(out_len);
for (size_t i = 0, j = 0; i < in_len;) {
uint32_t a = input[i] == '=' ? 0 & i++ : kDecodingTable[static_cast<int>(input[i++])];
uint32_t b = input[i] == '=' ? 0 & i++ : kDecodingTable[static_cast<int>(input[i++])];
uint32_t c = input[i] == '=' ? 0 & i++ : kDecodingTable[static_cast<int>(input[i++])];
uint32_t d = input[i] == '=' ? 0 & i++ : kDecodingTable[static_cast<int>(input[i++])];
for (size_t i = 0, j = 0; i < dataLen;) {
uint32_t a = dataPtr[i] == '=' ? 0 & i++ : kDecodingTable[static_cast<int>(dataPtr[i++])];
uint32_t b = dataPtr[i] == '=' ? 0 & i++ : kDecodingTable[static_cast<int>(dataPtr[i++])];
uint32_t c = dataPtr[i] == '=' ? 0 & i++ : kDecodingTable[static_cast<int>(dataPtr[i++])];
uint32_t d = dataPtr[i] == '=' ? 0 & i++ : kDecodingTable[static_cast<int>(dataPtr[i++])];
uint32_t triple = (a << 3 * 6) + (b << 2 * 6) + (c << 1 * 6) + (d << 0 * 6);
@@ -119,6 +126,10 @@ class Base64 {
return true;
}
static bool Decode(const std::string& input, std::string& out) {
return Decode(input.data(), input.length(), out);
}
};
#endif /* _BASE64_H_ */

63
headers/enums.h
View File

@@ -1,68 +1,7 @@
#ifndef _ENUMS_H_
#define _ENUMS_H_
typedef enum
{
BUTTON_LAYOUT_STICK,
BUTTON_LAYOUT_STICKLESS,
BUTTON_LAYOUT_BUTTONS_ANGLED,
BUTTON_LAYOUT_BUTTONS_BASIC,
BUTTON_LAYOUT_KEYBOARD_ANGLED,
BUTTON_LAYOUT_KEYBOARDA,
BUTTON_LAYOUT_DANCEPADA,
BUTTON_LAYOUT_TWINSTICKA,
BUTTON_LAYOUT_BLANKA,
BUTTON_LAYOUT_VLXA,
BUTTON_LAYOUT_FIGHTBOARD_STICK,
BUTTON_LAYOUT_FIGHTBOARD_MIRRORED,
BUTTON_LAYOUT_CUSTOMA,
} ButtonLayout;
typedef enum
{
BUTTON_LAYOUT_ARCADE,
BUTTON_LAYOUT_STICKLESSB,
BUTTON_LAYOUT_BUTTONS_ANGLEDB,
BUTTON_LAYOUT_VEWLIX,
BUTTON_LAYOUT_VEWLIX7,
BUTTON_LAYOUT_CAPCOM,
BUTTON_LAYOUT_CAPCOM6,
BUTTON_LAYOUT_SEGA2P,
BUTTON_LAYOUT_NOIR8,
BUTTON_LAYOUT_KEYBOARDB,
BUTTON_LAYOUT_DANCEPADB,
BUTTON_LAYOUT_TWINSTICKB,
BUTTON_LAYOUT_BLANKB,
BUTTON_LAYOUT_VLXB,
BUTTON_LAYOUT_FIGHTBOARD,
BUTTON_LAYOUT_FIGHTBOARD_STICK_MIRRORED,
BUTTON_LAYOUT_CUSTOMB,
} ButtonLayoutRight;
typedef enum
{
STATICSPLASH,
CLOSEIN,
CLOSEINCUSTOM,
NOSPLASH,
} SplashMode;
typedef enum
{
MAIN,
X,
Y,
Z,
CUSTOM,
LEGACY
} SplashChoice;
typedef enum
{
BOARD_LED_OFF,
MODE_INDICATOR,
INPUT_TEST
} OnBoardLedMode;
#include "enums.pb.h"
typedef enum
{

47
headers/gamepad.h
View File

@@ -5,9 +5,7 @@
#include <string.h>
#include "gamepad/GamepadDebouncer.h"
#include "gamepad/GamepadOptions.h"
#include "gamepad/GamepadState.h"
#include "gamepad/GamepadStorage.h"
#include "gamepad/descriptors/HIDDescriptors.h"
#include "gamepad/descriptors/SwitchDescriptors.h"
#include "gamepad/descriptors/XInputDescriptors.h"
@@ -16,6 +14,8 @@
#include "pico/stdlib.h"
#include "config.pb.h"
// MUST BE DEFINED FOR MPG
extern uint32_t getMillis();
extern uint64_t getMicro();
@@ -58,13 +58,7 @@ struct GamepadButtonMapping
class Gamepad {
public:
Gamepad(int debounceMS = 5, GamepadStorage *storage = &GamepadStore) :
debounceMS(debounceMS)
, f1Mask((GAMEPAD_MASK_S1 | GAMEPAD_MASK_S2))
, f2Mask((GAMEPAD_MASK_L3 | GAMEPAD_MASK_R3))
, debouncer(debounceMS)
, mpgStorage(storage)
{}
Gamepad(int debounceMS = 5);
void setup();
void process();
@@ -129,12 +123,17 @@ public:
inline bool __attribute__((always_inline)) pressedA2() { return pressedButton(GAMEPAD_MASK_A2); }
inline bool __attribute__((always_inline)) pressedF1() { return pressedButton(f1Mask); }
inline bool __attribute__((always_inline)) pressedF2() { return pressedButton(f2Mask); }
const GamepadOptions& getOptions() const { return options; }
void setInputMode(InputMode inputMode) { options.inputMode = inputMode; }
void setSOCDMode(SOCDMode socdMode) { options.socdMode = socdMode; }
void setDpadMode(DpadMode dpadMode) { options.dpadMode = dpadMode; }
GamepadDebouncer debouncer;
GamepadStorage *mpgStorage;
const uint8_t debounceMS;
uint16_t f1Mask;
uint16_t f2Mask;
GamepadOptions options;
GamepadState rawState;
GamepadState state;
GamepadButtonMapping *mapDpadUp;
@@ -158,9 +157,11 @@ public:
GamepadButtonMapping **gamepadMappings;
inline static const SOCDMode resolveSOCDMode(const GamepadOptions& options) {
return ((options.socdMode == SOCD_MODE_BYPASS) &&
(options.inputMode == INPUT_MODE_HID || options.inputMode == INPUT_MODE_SWITCH || options.inputMode == INPUT_MODE_PS4)) ?
SOCD_MODE_NEUTRAL : options.socdMode;
return (options.socdMode == SOCD_MODE_BYPASS &&
(options.inputMode == INPUT_MODE_HID ||
options.inputMode == INPUT_MODE_SWITCH ||
options.inputMode == INPUT_MODE_PS4)) ?
SOCD_MODE_NEUTRAL : options.socdMode;
};
private:
@@ -168,14 +169,16 @@ private:
void pressKey(uint8_t code);
uint8_t getModifier(uint8_t code);
GamepadHotkeyEntry hotkeyF1Up;
GamepadHotkeyEntry hotkeyF1Down;
GamepadHotkeyEntry hotkeyF1Left;
GamepadHotkeyEntry hotkeyF1Right;
GamepadHotkeyEntry hotkeyF2Up;
GamepadHotkeyEntry hotkeyF2Down;
GamepadHotkeyEntry hotkeyF2Left;
GamepadHotkeyEntry hotkeyF2Right;
GamepadOptions& options;
HotkeyEntry hotkeyF1Up;
HotkeyEntry hotkeyF1Down;
HotkeyEntry hotkeyF1Left;
HotkeyEntry hotkeyF1Right;
HotkeyEntry hotkeyF2Up;
HotkeyEntry hotkeyF2Down;
HotkeyEntry hotkeyF2Left;
HotkeyEntry hotkeyF2Right;
};
#endif

2
headers/gamepad/GamepadDescriptors.h
View File

@@ -13,6 +13,8 @@
#include "descriptors/KeyboardDescriptors.h"
#include "descriptors/PS4Descriptors.h"
#include "enums.pb.h"
// Default value used for networking, override if necessary
static uint8_t macAddress[6] = { 0x02, 0x02, 0x84, 0x6A, 0x96, 0x00 };

50
headers/gamepad/GamepadEnums.h
View File

@@ -5,38 +5,6 @@
#pragma once
// The available input modes
// TODO: Think about making enums "class enum" instead. This will keep them out of the global namespace
// and provide strong type assurance instead of being treated as ints. Also, modern c++ would tend towards
// using Pascal-case naming for the individual declarations.
typedef enum
{
INPUT_MODE_XINPUT,
INPUT_MODE_SWITCH,
INPUT_MODE_HID,
INPUT_MODE_KEYBOARD,
INPUT_MODE_PS4,
INPUT_MODE_CONFIG = 255,
} InputMode;
// The available stick emulation modes
typedef enum
{
DPAD_MODE_DIGITAL,
DPAD_MODE_LEFT_ANALOG,
DPAD_MODE_RIGHT_ANALOG,
} DpadMode;
// The available SOCD cleaning methods
typedef enum
{
SOCD_MODE_UP_PRIORITY, // U+D=U, L+R=N
SOCD_MODE_NEUTRAL, // U+D=N, L+R=N
SOCD_MODE_SECOND_INPUT_PRIORITY, // U>D=D, L>R=R (Last Input Priority, aka Last Win)
SOCD_MODE_FIRST_INPUT_PRIORITY, // U>D=U, L>R=L (First Input Priority, aka First Win)
SOCD_MODE_BYPASS, // U+D=UD, L+R=LR (No cleaning applied)
} SOCDMode;
// Enum for tracking last direction state of Second Input SOCD method
typedef enum
{
@@ -46,21 +14,3 @@ typedef enum
DIRECTION_LEFT,
DIRECTION_RIGHT
} DpadDirection;
// The available hotkey actions
typedef enum
{
HOTKEY_NONE,
HOTKEY_DPAD_DIGITAL,
HOTKEY_DPAD_LEFT_ANALOG,
HOTKEY_DPAD_RIGHT_ANALOG,
HOTKEY_HOME_BUTTON,
HOTKEY_CAPTURE_BUTTON,
HOTKEY_SOCD_UP_PRIORITY,
HOTKEY_SOCD_NEUTRAL,
HOTKEY_SOCD_LAST_INPUT,
HOTKEY_INVERT_X_AXIS,
HOTKEY_INVERT_Y_AXIS,
HOTKEY_SOCD_FIRST_INPUT,
HOTKEY_SOCD_BYPASS
} GamepadHotkey;

56
headers/gamepad/GamepadOptions.h
View File

@@ -1,56 +0,0 @@
/*
* SPDX-License-Identifier: MIT
* SPDX-FileCopyrightText: Copyright (c) 2021 Jason Skuby (mytechtoybox.com)
*/
#pragma once
#include <stdint.h>
#include "GamepadEnums.h"
typedef struct
{
uint8_t dpadMask;
GamepadHotkey action;
} GamepadHotkeyEntry;
struct GamepadOptions
{
InputMode inputMode {InputMode::INPUT_MODE_XINPUT};
DpadMode dpadMode {DpadMode::DPAD_MODE_DIGITAL};
SOCDMode socdMode {SOCDMode::SOCD_MODE_NEUTRAL};
bool invertXAxis;
bool invertYAxis;
uint8_t keyDpadUp;
uint8_t keyDpadDown;
uint8_t keyDpadLeft;
uint8_t keyDpadRight;
uint8_t keyButtonB1;
uint8_t keyButtonB2;
uint8_t keyButtonB3;
uint8_t keyButtonB4;
uint8_t keyButtonL1;
uint8_t keyButtonR1;
uint8_t keyButtonL2;
uint8_t keyButtonR2;
uint8_t keyButtonS1;
uint8_t keyButtonS2;
uint8_t keyButtonL3;
uint8_t keyButtonR3;
uint8_t keyButtonA1;
uint8_t keyButtonA2;
GamepadHotkeyEntry hotkeyF1Up;
GamepadHotkeyEntry hotkeyF1Down;
GamepadHotkeyEntry hotkeyF1Left;
GamepadHotkeyEntry hotkeyF1Right;
GamepadHotkeyEntry hotkeyF2Up;
GamepadHotkeyEntry hotkeyF2Down;
GamepadHotkeyEntry hotkeyF2Left;
GamepadHotkeyEntry hotkeyF2Right;
bool switchTpShareForDs4;
uint32_t checksum;
};

1
headers/gamepad/GamepadState.h
View File

@@ -7,6 +7,7 @@
#include <stdint.h>
#include "GamepadEnums.h"
#include "enums.pb.h"
#define GAMEPAD_BUTTON_COUNT 14

24
headers/gamepad/GamepadStorage.h
View File

@@ -1,24 +0,0 @@
/*
* SPDX-License-Identifier: MIT
* SPDX-FileCopyrightText: Copyright (c) 2021 Jason Skuby (mytechtoybox.com)
*/
#pragma once
#include <stdint.h>
#include "GamepadOptions.h"
#define STORAGE_FIRST_AVAILBLE_INDEX 2048
class GamepadStorage
{
public:
virtual void start(); // TODO: Should be pure virtual.
virtual void save(); // TODO: Should be pure virtual.
GamepadOptions getGamepadOptions();
void setGamepadOptions(GamepadOptions options);
};
static GamepadStorage GamepadStore;

2
headers/helper.h
View File

@@ -53,4 +53,6 @@ const std::string BUTTON_LABEL_A2 = "A2";
#define PLED_COLOR ColorWhite // White
#endif
static inline bool isValidPin(int32_t pin) { return pin >= 0 && pin < NUM_BANK0_GPIOS; }
#endif

273
headers/storagemanager.h
View File

@@ -14,215 +14,11 @@
#include "helper.h"
#include "gamepad.h"
#include "mbedtls/rsa.h"
#include "config.pb.h"
#define GAMEPAD_STORAGE_INDEX 0 // 1024 bytes for gamepad options
#define BOARD_STORAGE_INDEX 1024 // 512 bytes for hardware options
#define LED_STORAGE_INDEX 1536 // 512 bytes for LED configuration
#define ANIMATION_STORAGE_INDEX 2048 // 1024 bytes for LED animations
#define ADDON_STORAGE_INDEX 3072 // 1024 bytes for Add-Ons
#define PS4_STORAGE_INDEX 4096 // 2048 bytes for PS4 options
#define SPLASH_IMAGE_STORAGE_INDEX 6144 // 1032 bytes for Display Config
#include <atomic>
#define CHECKSUM_MAGIC 0 // Checksum CRC
#define NOCHECKSUM_MAGIC 0xDEADBEEF // No checksum CRC
struct ButtonLayoutParams
{
union {
ButtonLayout layout;
ButtonLayoutRight layoutRight;
};
int startX;
int startY;
int buttonRadius;
int buttonPadding;
};
struct ButtonLayoutCustomOptions
{
ButtonLayoutParams params;
ButtonLayoutParams paramsRight;
}; // 76 bytes
struct BoardOptions
{
bool hasBoardOptions;
uint8_t pinDpadUp;
uint8_t pinDpadDown;
uint8_t pinDpadLeft;
uint8_t pinDpadRight;
uint8_t pinButtonB1;
uint8_t pinButtonB2;
uint8_t pinButtonB3;
uint8_t pinButtonB4;
uint8_t pinButtonL1;
uint8_t pinButtonR1;
uint8_t pinButtonL2;
uint8_t pinButtonR2;
uint8_t pinButtonS1;
uint8_t pinButtonS2;
uint8_t pinButtonL3;
uint8_t pinButtonR3;
uint8_t pinButtonA1;
uint8_t pinButtonA2;
ButtonLayout buttonLayout;
ButtonLayoutRight buttonLayoutRight;
SplashMode splashMode;
SplashChoice splashChoice;
int splashDuration; // -1 = Always on
uint8_t i2cSDAPin;
uint8_t i2cSCLPin;
int i2cBlock;
uint32_t i2cSpeed;
bool hasI2CDisplay;
int displayI2CAddress;
uint8_t displaySize;
uint8_t displayFlip;
bool displayInvert;
int displaySaverTimeout;
ButtonLayoutCustomOptions buttonLayoutCustomOptions;
char boardVersion[32]; // 32-char limit to board name
uint32_t checksum;
};
struct AddonOptions {
uint8_t pinButtonTurbo;
uint8_t pinButtonReverse;
uint8_t pinSliderLS;
uint8_t pinSliderRS;
uint8_t pinSliderSOCDOne;
uint8_t pinSliderSOCDTwo;
uint8_t turboShotCount; // Turbo
uint8_t pinTurboLED; // Turbo LED
uint8_t pinReverseLED; // Reverse LED
uint8_t reverseActionUp;
uint8_t reverseActionDown;
uint8_t reverseActionLeft;
uint8_t reverseActionRight;
uint8_t i2cAnalog1219SDAPin;
uint8_t i2cAnalog1219SCLPin;
int i2cAnalog1219Block;
uint32_t i2cAnalog1219Speed;
uint8_t i2cAnalog1219Address;
uint8_t pinDualDirUp; // Pins for Dual Directional Input
uint8_t pinDualDirDown;
uint8_t pinDualDirLeft;
uint8_t pinDualDirRight;
DpadMode dualDirDpadMode; // LS/DP/RS
uint8_t dualDirCombineMode; // Mix/Gamepad/Dual/None
OnBoardLedMode onBoardLedMode;
uint8_t analogAdcPinX;
uint8_t analogAdcPinY;
uint16_t bootselButtonMap;
uint8_t extraButtonPin;
uint32_t extraButtonMap;
uint8_t buzzerPin;
uint8_t buzzerVolume;
uint8_t playerNumber;
uint8_t shmupMode; // Turbo SHMUP Mode
uint8_t shmupMixMode; // How we mix turbo and non-turbo buttons
uint16_t shmupAlwaysOn1;
uint16_t shmupAlwaysOn2;
uint16_t shmupAlwaysOn3;
uint16_t shmupAlwaysOn4;
uint8_t pinShmupBtn1;
uint8_t pinShmupBtn2;
uint8_t pinShmupBtn3;
uint8_t pinShmupBtn4;
uint16_t shmupBtnMask1;
uint16_t shmupBtnMask2;
uint16_t shmupBtnMask3;
uint16_t shmupBtnMask4;
uint8_t pinShmupDial;
SOCDMode sliderSOCDModeOne;
SOCDMode sliderSOCDModeTwo;
SOCDMode sliderSOCDModeDefault;
uint8_t wiiExtensionSDAPin;
uint8_t wiiExtensionSCLPin;
int wiiExtensionBlock;
uint32_t wiiExtensionSpeed;
uint8_t snesPadLatchPin;
uint8_t snesPadClockPin;
uint8_t snesPadDataPin;
uint8_t AnalogInputEnabled;
uint8_t BoardLedAddonEnabled;
uint8_t BootselButtonAddonEnabled;
uint8_t BuzzerSpeakerAddonEnabled;
uint8_t DualDirectionalInputEnabled;
uint8_t ExtraButtonAddonEnabled;
uint8_t I2CAnalog1219InputEnabled;
//bool I2CDisplayAddonEnabled; // I2C is special case
uint8_t JSliderInputEnabled;
//bool NeoPicoLEDAddonEnabled; // NeoPico is special case
//bool PlayerLEDAddonEnabled; // PlayerLED is special case
uint8_t PlayerNumAddonEnabled;
uint8_t PS4ModeAddonEnabled;
uint8_t ReverseInputEnabled;
uint8_t TurboInputEnabled;
uint8_t SliderSOCDInputEnabled;
uint8_t WiiExtensionAddonEnabled;
uint8_t SNESpadAddonEnabled;
uint32_t checksum;
};
struct SplashImage {
uint8_t data[16*64];
uint32_t checksum;
};
struct PS4Options {
uint8_t serial[16];
uint8_t signature[256];
mbedtls_mpi_uint rsa_n[64];
mbedtls_mpi_uint rsa_e[1];
mbedtls_mpi_uint rsa_d[64];
mbedtls_mpi_uint rsa_p[32];
mbedtls_mpi_uint rsa_q[32];
mbedtls_mpi_uint rsa_dp[32];
mbedtls_mpi_uint rsa_dq[32];
mbedtls_mpi_uint rsa_qp[32];
mbedtls_mpi_uint rsa_rn[64];
uint32_t checksum;
};
struct LEDOptions
{
bool useUserDefinedLEDs;
int dataPin;
LEDFormat ledFormat;
ButtonLayout ledLayout;
uint8_t ledsPerButton;
uint8_t brightnessMaximum;
uint8_t brightnessSteps;
int indexUp;
int indexDown;
int indexLeft;
int indexRight;
int indexB1;
int indexB2;
int indexB3;
int indexB4;
int indexL1;
int indexR1;
int indexL2;
int indexR2;
int indexS1;
int indexS2;
int indexL3;
int indexR3;
int indexA1;
int indexA2;
int pledType;
int pledPin1;
int pledPin2;
int pledPin3;
int pledPin4;
RGB pledColor;
uint32_t checksum;
};
#include "pico/critical_section.h"
#define SI Storage::getInstance()
@@ -237,25 +33,23 @@ public:
return instance;
}
void setBoardOptions(BoardOptions); // Board Options
const BoardOptions& getBoardOptions() { return boardOptions; }
Config& getConfig() { return config; }
GamepadOptions& getGamepadOptions() { return config.gamepadOptions; }
HotkeyOptions& getHotkeyOptions() { return config.hotkeyOptions; }
PinMappings& getPinMappings() { return config.pinMappings; }
KeyboardMapping& getKeyboardMapping() { return config.keyboardMapping; }
DisplayOptions& getDisplayOptions() { return config.displayOptions; }
DisplayOptions& getPreviewDisplayOptions() { return previewDisplayOptions; }
LEDOptions& getLedOptions() { return config.ledOptions; }
AddonOptions& getAddonOptions() { return config.addonOptions; }
AnimationOptions_Proto& getAnimationOptions() { return config.animationOptions; }
void setPreviewBoardOptions(const BoardOptions&); // Preview Board Options
const BoardOptions& getPreviewBoardOptions() { return previewBoardOptions; }
bool save();
void setAddonOptions(AddonOptions); // Add-On Options
const AddonOptions& getAddonOptions() { return addonOptions; }
// Perform saves that were enqueued from core1
void performEnqueuedSaves();
void setSplashImage(const SplashImage&);
const SplashImage& getSplashImage() { return splashImage; }
void setLEDOptions(LEDOptions); // LED Options
void setDefaultLEDOptions();
const LEDOptions& getLEDOptions() { return ledOptions; }
void savePS4Options(); // PS4 Options
void setDefaultPS4Options();
PS4Options * getPS4Options();
void enqueueAnimationOptionsSave(const AnimationOptions& animationOptions);
void SetConfigMode(bool); // Config Mode (on-boot)
bool GetConfigMode();
@@ -281,34 +75,27 @@ public:
const int * getPLEDPins() { return pledPins; }
private:
Storage() : gamepad(0) {
EEPROM.start(); // init EEPROM
initBoardOptions();
initAddonOptions();
initLEDOptions();
initSplashImage();
initPS4Options();
}
Storage();
void initBoardOptions();
void initPreviewBoardOptions();
void initAddonOptions();
void initLEDOptions();
void initSplashImage();
void setDefaultBoardOptions();
void setDefaultAddonOptions();
void setDefaultSplashImage();
void initPS4Options();
bool CONFIG_MODE; // Config mode (boot)
Gamepad * gamepad; // Gamepad data
Gamepad * processedGamepad; // Gamepad with ONLY processed data
BoardOptions boardOptions;
BoardOptions previewBoardOptions;
AddonOptions addonOptions;
LEDOptions ledOptions;
PS4Options ps4Options;
bool CONFIG_MODE = false; // Config mode (boot)
Gamepad * gamepad = nullptr; // Gamepad data
Gamepad * processedGamepad = nullptr; // Gamepad with ONLY processed data
uint8_t featureData[32]; // USB X-Input Feature Data
SplashImage splashImage;
DisplayOptions previewDisplayOptions;
Config config;
std::atomic<bool> animationOptionsSavePending;
critical_section_t animationOptionsCs;
uint32_t animationOptionsCrc = 0;
AnimationOptions animationOptionsToSave = {};
int pledPins[4];
};

2
headers/themes.h
View File

@@ -11,6 +11,8 @@
#include "AnimationStation.hpp"
#include "helper.h"
#include "config.pb.h"
using namespace std;
static map<uint32_t, RGB> themeStaticRainbow({

2
lib/AnimationStation/src/Animation.hpp
View File

@@ -38,7 +38,7 @@ struct RGB {
}
}
inline uint32_t value(LEDFormat format, float brightnessX = 1.0F) {
inline uint32_t value(LEDFormat format, float brightnessX = 1.0F) const {
switch (format) {
case LED_FORMAT_GRB:
return ((uint32_t)(g * brightnessX) << 16)

2
lib/AnimationStation/src/AnimationStorage.hpp
View File

@@ -7,9 +7,7 @@ class AnimationStorage
{
public:
void save();
AnimationOptions getAnimationOptions();
void setAnimationOptions(AnimationOptions options);
};
static AnimationStorage AnimationStore;

1
lib/CMakeLists.txt
View File

@@ -5,6 +5,7 @@ add_subdirectory(CRC32)
add_subdirectory(FlashPROM)
add_subdirectory(httpd)
add_subdirectory(lwip-port)
add_subdirectory(nanopb)
add_subdirectory(NeoPico)
add_subdirectory(OneBitDisplay)
add_subdirectory(PlayerLEDs)

23
lib/FlashPROM/src/FlashPROM.cpp
View File

@@ -5,7 +5,7 @@
#include "FlashPROM.h"
uint8_t FlashPROM::cache[EEPROM_SIZE_BYTES] = { };
uint8_t FlashPROM::writeCache[EEPROM_SIZE_BYTES];
volatile static alarm_id_t flashWriteAlarm = 0;
volatile static spin_lock_t *flashLock = nullptr;
@@ -32,22 +32,7 @@ void FlashPROM::start()
if (flashLock == nullptr)
flashLock = spin_lock_instance(spin_lock_claim_unused(true));
memcpy(cache, reinterpret_cast<uint8_t *>(EEPROM_ADDRESS_START), EEPROM_SIZE_BYTES);
// When flash is new/reset, all bits are set to 1.
// If all bits from the FlashPROM section are 1's then set to 0's.
bool reset = true;
for (int i = 0; i < EEPROM_SIZE_BYTES; i++)
{
if (cache[i] != 0xFF)
{
reset = false;
break;
}
}
if (reset)
this->reset();
memcpy(writeCache, reinterpret_cast<uint8_t *>(EEPROM_ADDRESS_START), EEPROM_SIZE_BYTES);
}
/* We don't have an actual EEPROM, so we need to be extra careful about minimizing writes. Instead
@@ -58,11 +43,11 @@ void FlashPROM::commit()
while (is_spin_locked(flashLock));
if (flashWriteAlarm != 0)
cancel_alarm(flashWriteAlarm);
flashWriteAlarm = add_alarm_in_ms(EEPROM_WRITE_WAIT, writeToFlash, cache, true);
flashWriteAlarm = add_alarm_in_ms(EEPROM_WRITE_WAIT, writeToFlash, writeCache, true);
}
void FlashPROM::reset()
{
memset(cache, 0, EEPROM_SIZE_BYTES);
memset(writeCache, 0, EEPROM_SIZE_BYTES);
commit();
}

21
lib/FlashPROM/src/FlashPROM.h
View File

@@ -25,26 +25,7 @@ class FlashPROM
void commit();
void reset();
template<typename T>
T &get(uint16_t const index, T &value)
{
if (index < EEPROM_SIZE_BYTES)
memcpy(&value, &cache[index], sizeof(T));
return value;
}
template<typename T>
void set(uint16_t const index, const T &value)
{
uint16_t size = sizeof(T);
if ((index + size) <= EEPROM_SIZE_BYTES)
memcpy(&cache[index], &value, sizeof(T));
}
private:
static uint8_t cache[EEPROM_SIZE_BYTES];
static uint8_t writeCache[EEPROM_SIZE_BYTES];
};
static FlashPROM EEPROM;

7
lib/PlayerLEDs/src/PlayerLEDs.h
View File

@@ -10,13 +10,6 @@
#define PLED_MAX_BRIGHTNESS 0xFF
#define PLED_MAX_LEVEL 0xFFFF
typedef enum
{
PLED_TYPE_NONE = -1,
PLED_TYPE_PWM = 0,
PLED_TYPE_RGB = 1,
} PLEDType;
typedef enum
{
PLED_STATE_LED1 = (1 << 0),

12
lib/TinyUSB_Gamepad/CMakeLists.txt
View File

@@ -1,3 +1,6 @@
include(../../compile_proto.cmake)
compile_proto()
add_library(TinyUSB_Gamepad
src/hid_driver.cpp
src/net_driver.cpp
@@ -5,16 +8,19 @@ src/tusb_driver.cpp
src/usb_descriptors.cpp
src/xinput_driver.cpp
src/ps4_driver.cpp
${PROTO_OUTPUT_DIR}/enums.pb.h
)
target_include_directories(TinyUSB_Gamepad PUBLIC
src
target_include_directories(TinyUSB_Gamepad PUBLIC
${CMAKE_CURRENT_SOURCE_DIR}/src
${CMAKE_SOURCE_DIR}/headers
${CMAKE_SOURCE_DIR}/lib/CRC32/src
${CMAKE_SOURCE_DIR}/lib/mbedtls/include
${CMAKE_SOURCE_DIR}/lib/nanopb
${PROTO_OUTPUT_DIR}
)
target_link_libraries(TinyUSB_Gamepad
pico_stdlib
pico_mbedtls
tinyusb_device
rndis
)
)

2
lib/TinyUSB_Gamepad/src/hid_driver.cpp
View File

@@ -11,6 +11,8 @@
#include "class/hid/hid_device.h"
#include "gamepad/GamepadDescriptors.h"
#include "enums.pb.h"
// Magic byte sequence to enable PS button on PS3
static const uint8_t magic_init_bytes[8] = {0x21, 0x26, 0x01, 0x07, 0x00, 0x00, 0x00, 0x00};

1
lib/TinyUSB_Gamepad/src/usb_driver.h
View File

@@ -6,6 +6,7 @@
#pragma once
#include "gamepad/GamepadDescriptors.h"
#include "enums.pb.h"
typedef enum
{

33
lib/nanopb/.gitignore vendored Normal file
View File

@@ -0,0 +1,33 @@
*.gcda
*.gcno
*.gcov
*.o
*.pb.c
*.pb.h
*.pb
*.pyc
*_pb2.py
*~
*.tar.gz
*.swp
.sconsign.dblite
config.log
.sconf_temp
tests/build
julkaisu.txt
dist
docs/*.html
docs/generator_flow.png
examples/simple/simple
examples/network_server/client
examples/network_server/server
examples/using_double_on_avr/decode_double
examples/using_double_on_avr/encode_double
examples/using_double_on_avr/test_conversions
examples/using_union_messages/decode
examples/using_union_messages/encode
generator/nanopb_pb2.pyc
!generator-bin/**/*
bazel-*
extra/poetry/build
build/

123
lib/nanopb/AUTHORS.txt Normal file
View File

@@ -0,0 +1,123 @@
Petteri Aimonen <jpa@npb.mail.kapsi.fi>
Michael Poole <mdpoole@troilus.org>
Daniel Kan <extremeblue99@gmail.com>
Stan Hu <stanhu@aclimalabs.com>
David Hotham <david.hotham@blueyonder.co.uk>
Steffen Siering <steffen siering gmail com>
Jens Steinhauser <jens.steinhauser@gmail.com>
Pavel Ilin <ilin.pa@gmail.com>
Kent Ryhorchuk <kryhorchuk@xeralux.com>
Martin Donath <scifish@gmail.com>
Oliver Lee <oliverzlee@gmail.com>
Michael Haberler <git@mah.priv.at>
Nicolas Colomer <ncolomer@viadeoteam.com>
Ivan Kravets <me@ikravets.com>
Kyle Manna <kyle@kylemanna.com>
Benjamin Kamath <ben.kamath@synapse.com>
Andrew Ruder <andrew.ruder@elecsyscorp.com>
Kenshi Kawaguchi <kenshi@recurse.ca>
isotes <isotes@gmail.com>
Maxim Khitrov <max@mxcrypt.com>
Yaniv Mordekhay <yanivmo@users.noreply.github.com>
Ming Zhao <mzhao@luminatewireless.com>
Google, Inc.
Tom Roeder <tmroeder@google.com>
Piotr Sikora <piotrsikora@google.com>
Bernhard Krämer <bdkrae@gmail.com>
Konstantin Podsvirov <konstantin@podsvirov.pro>
William A. Kennington III <wak@google.com>
Guillaume Lager <g.lager@innoseis.com>
Tobias Haegermarck <tobias.haegermarck@gmail.com>
Justin DeMartino <jdemarti@gmail.com>
Constantine Grantcharov <cgrantcharov@trustpointinnovation.com>
Nick Ewalt <nicholasewalt@google.com>
Harald Fernengel <harryf@gmx.com>
Alice Wang <aw@squareup.com>
Kevin Fitch <kfitch42@gmail.com>
Kamal Marhubi <kamal@marhubi.com>
Elco Jacobs <elco@brewpi.com>
Sébastien Morin <sebastien.morin@primerogames.com>
Dave Flogeras <dflogeras2@gmail.com>
Edward Z. Yang <ezyang@mit.edu>
Robbie Shade <rjshade@google.com>
Andrew Ballinger <andrewballinger@stratisopt.com>
Hamina, Juha-Pekka <Juha-Pekka.Hamina@nordicsemi.no>
Jason Bishop <jason.bishop@bigassfans.com>
matejcik <ja@matejcik.cz>
Tobias Müller <Tobias_Mueller@twam.info>
Jari Vetoniemi <mailroxas@gmail.com>
Gabriel Staples <ercaguy@gmail.com>
Amarnath <amarnath.h.96@gmail.com>
Michal Rostecki <mrostecki@suse.de>
Pei Wang <wangpei10@baidu.com>
Noah Pendleton <2538614+noahp@users.noreply.github.com>
Pavol Rusnak <pavol@rusnak.io>
der-imp <falkjan@msn.com>
Mark Hill <markleehill@gmail.com>
Torfinn Berset <torfinn@bloom-life.com>
Bo Langgaard Lind <bo.langgaard.lind@gmail.com>
Stephane Dorre <stephane.dorre@cobi.bike>
Phillip Cao <Phillip.Cao@fphcare.co.nz>
Melvin Wang <melvin.mc.wang@gmail.com>
Joshua Salzedo <thHunkn0WNd@gmail.com>
Adam Klama <klama.adam@gmail.com>
Anton Matosov <amatosov@roblox.com>
berni155 <bdkrae@gmail.com>
David Lin <dtwlin@google.com>
devjoa <devjoa@gmail.com>
Evan Fisher <schleb@gmail.com>
Fay <fay2003hiend@gmail.com>
Florian Märkl <info@florianmaerkl.de>
Franck <franck.sehedic@ledger.fr>
Ilya Averyanov <i.averyanov@geoscan.aero>
John Ullman <jrullman@google.com>
Ket3r <peter.kempter@gmail.com>
maciej <maciej.matuszak@gmail.com>
Marek Zukal <marek.zukal@gmail.com>
Paul Beusterien <paulbeusterien@google.com>
Rogers Guedes <rogers.guedes@smdautomacao.com>
Stefan R. Filipek <srfilipek@gmail.com>
T. Carl Beery <beeryt@users.noreply.github.com>
Vitali Lovich <vlovich@google.com>
Vojtěch Boček <vbocek@gmail.com>
Wael Nasreddine <wael.nasreddine@gmail.com>
wangli28 <wangli28@beyondsoft.com>
Zukaitis <gediminas.zukaitis@office.auriga.msk>
Alex Pacini <alexpacini90@gmail.com>
Cong <congusbongus@gmail.com>
kurddt <kurddt@users.noreply.github.com>
otopetrik <oto.petrik@gmail.com>
Psy-Kai <psykai1993@googlemail.com>
a1lu <a1lu@users.noreply.github.com>
L <46594312+WakandaO2@users.noreply.github.com>
Melvin Wang <mwang@sibros.tech>
Tim Gates <tim.gates@iress.com>
leabut <leabut@users.noreply.github.com>
Angel ILIEV <a.v.iliev13@gmail.com>
Jakub Tymejczyk <jakub@tymejczyk.pl>
Matthew Simmons <simmonmt@acm.org>
Anthony Pesch <inolen@gmail.com>
Avik De <avikde@gmail.com>
Conrad Wood <github@conradwood.net>
David Sabatie <david.sabatie@notrenet.com>
Sebastian Stockhammer <sebastian.stockhammer@rosenberger.de>
Gil Shapira <gil.shapira@intusurg.com>
Ian Frosst <ianjfrosst@gmail.com>
Ingo Kresse <ingo.kresse@kuka.com>
Ivan Zrno <ivan.zrno2@gmail.com>
Jonathan Seilkopf <j.seilkopf@isatech.de>
Karl Ljungkvist <k.ljungkvist@gmail.com>
Mathis Logemann <mathisloge@gmail.com>
Oleg Dolgy <60554929+odolgy@users.noreply.github.com>
Pavel Sokolov <pavel@sokolov.me>
Slavey Karadzhov <slav@attachix.com>
Tobias Nießen <tniessen@tnie.de>
Christian Balcom <robot.inventor@gmail.com>
Christopher Hughes <67643395+chughes-pika@users.noreply.github.com>
Greg Balke <gbalke@berkeley.edu>
Jussi Keränen <jussike@gmail.com>
Krzysztof Rosinski <krzysiek@jrdltd.co.uk>
Nathaniel Brough <nathaniel.brough@gmail.com>
Sean Kahler <itsbattledash@gmail.com>
Valerii Koval <valeros@users.noreply.github.com>
Gediminas Žukaitis <gediminas.zukaitis@protonmail.com>

92
lib/nanopb/BUILD.bazel Normal file
View File

@@ -0,0 +1,92 @@
load("@rules_proto//proto:defs.bzl", "proto_library")
load("@rules_python//python:defs.bzl", "py_binary")
load("@nanopb_pypi//:requirements.bzl", "requirement")
load("@rules_proto_grpc//:defs.bzl", "proto_plugin")
load("@bazel_skylib//rules:copy_file.bzl", "copy_file")
load("//extra/bazel:nanopb_cc_proto_library.bzl", "cc_nanopb_proto_library")
load("@rules_python//python/pip_install:requirements.bzl", "compile_pip_requirements")
package(default_visibility = ["//visibility:public"])
licenses(["notice"])
exports_files(["LICENSE.txt"])
cc_library(
name = "nanopb",
srcs = [
"pb_common.c",
"pb_decode.c",
"pb_encode.c",
],
hdrs = [
"pb.h",
"pb_common.h",
"pb_decode.h",
"pb_encode.h",
],
visibility = ["//visibility:public"],
)
copy_file(
name = "protoc-gen-nanopb.py",
src = "generator/protoc-gen-nanopb",
out = "generator/protoc-gen-nanopb.py",
allow_symlink = True,
)
py_binary(
name = "protoc-gen-nanopb",
srcs = [
"generator/nanopb_generator.py",
":protoc-gen-nanopb.py",
],
deps = [
requirement("grpcio-tools"),
],
)
proto_plugin(
name = "nanopb_plugin",
options = [
'--library-include-format=quote',
],
outputs = [
"{protopath}.pb.h",
"{protopath}.pb.c",
],
separate_options_flag = True,
tool = ":protoc-gen-nanopb",
visibility = ["//visibility:public"],
)
proto_library(
name = "descriptor",
srcs = [
"generator/proto/google/protobuf/descriptor.proto",
],
strip_import_prefix = "generator/proto/",
)
proto_library(
name = "nanopb_proto",
srcs = [
"generator/proto/nanopb.proto",
],
strip_import_prefix = "generator/proto/",
deps = [":descriptor"],
)
cc_nanopb_proto_library(
name = "test_compilation",
protos = [":descriptor"],
visibility = ["//visibility:private"],
)
# Run `bazel run //:requirements.update` if you want to update the requirements.
compile_pip_requirements(
name = "requirements",
extra_args = ["--allow-unsafe"],
requirements_in = "extra/requirements.txt",
requirements_txt = "extra/requirements_lock.txt",
)

597
lib/nanopb/CHANGELOG.txt Normal file
View File

@@ -0,0 +1,597 @@
nanopb-0.4.7 (2022-12-11)
Fix comments generation for submessages (#788)
Fix handling of spaces in arguments passed through protoc (#810)
Fix problems with multiple files and mangle_names option (#783, #820)
Fix generator error when using M_STRIP_PACKAGE without package name (#795)
Fix compilation error with fixed size array and FT_POINTER (#630)
Fix wrong format in Python Poetry project file (#811)
Fix unnecessary generator message when using --quiet (#831)
Fix enum_to_string with C++ (#838)
Fix /* */ inside .proto file comment
Workaround python-protobuf version issues (#787)
Safeguard substraction in pb_read() with custom streams (#697)
Always include pb_release() as function, instead of macro. (#802)
Allow using = instead of : with generator option -s
Allow specifying include format without special characters (#810)
Allow including headers from inside of extern C (#814)
Add option NANOPB_PB2_TEMP_DIR to store nanopb_pb2.py in a temporary dir (#601)
Add compile-time error message for when PB_FIELD_32BIT is needed (#680, #827)
Add --c-style command line option for naming style (#199, #533, #791)
Add --protoc-opt to nanopb_generator.py (#628)
Add ENUMTYPE convenience macros (#803)
Add Bazel build rules (#360, #500)
Generator: keep order of messages when possible
Test case improvements (#792)
PlatformIO build rule improvements (#808, #809, #819, #834, #839, #840)
CMake build rule improvements (#822)
CMakeLists: use protoc wrapper script by default (#769)
nanopb-0.4.6 (2022-05-30)
Fix passing of error message from substream callback (#703)
Fix comments going to wrong member variables (#701)
Fix regression in 0.4.3 where generator did not find all dependencies (#720)
Fix FindNanopb.cmake not finding options file (#659)
Fix double-definition errors with size_union (#692)
Fix generator error with same inner message name (#746)
Fix infinite recursion in generator/protoc script (#762)
Fix unicode comment handling for Python 2 (#740)
Fix compiler warnings with PB_BUFFER_ONLY (#717)
Fix options dependency in nanopb.mk (#666)
Fix handling of filenames with dot in them in FindNanopb.cmake (#756)
Add fallback_type option (#772, #773)
Use C11 static assert mechanism by default (#761, #766)
Use 'static_assert' keyword for iar (#679)
Explicitly check for pItem == NULL to satisfy Xcode analyzer (#667, #674)
Support --proto-path as alias to -I (#749)
Refactor name mangling to separate class, improve error messages (#735)
Move PB_WT_PACKED definition to the header to fix compiler warnings (#671)
FindNanopb.cmake: use --nanopb_opt for option passing by default (#752)
FindNanopb.cmake: Add option NANOPB_GENERATE_CPP_STANDALONE (#741)
FindNanopb.cmake: Add PROTOC_OPTIONS variable (#768, #771)
CMakeLists: add build interface for using as a submodule (#669)
CMakeLists: fix error with nanopb_BUILD_GENERATOR=OFF (#764)
CMakeLists: make more uniform (#676)
CMakeLists: Fix uninitialized PYTHON_INSTDIR (#652)
Clean up CMake examples (#741)
Rebuild nanopb_pb2.py and print version numbers on import failure (#733, #742)
Use memcpy instead of iterating on buf_read/write (#751)
Add generator support for PlatformIO (#718)
Add clean target to generator/proto/Makefile (#681)
Windows .bats: use standard python invocation instead of py.exe launcher (#657)
Fix problems running tests with newer SCons version
Improve handling of varint overflows
Improve optimization for little-endian platforms
NOTE: During development, prereleases were published on PlatformIO registry
as versions 0.4.6 - 0.4.6.3. The version 0.4.6.4 on PlatformIO corresponds
to the real final 0.4.6 release.
nanopb-0.4.5 (2021-03-22)
Fix invalid free() with oneof (#647, GHSA-7mv5-5mxh-qg88)
Fix unordered field numbers inside oneof causing fields to be ignored (#617)
Fix pb_decode() not initializing fields inside oneof (#635
Fix compiler errors with complex oneof hierarchy and sizeof() (#610)
Fix descriptor width calculation for 64-bit types (#644)
Fix compiler error in generated initializer for submessage callback (#631)
Fix duplicate union definition in generated file (#637)
Fix test case build error on SCons 4.1.0
Pip package: include nanopb_pb2.py (#629)
Make generator consider dependencies recursively (#610)
Bazel build system updates (#633)
Add support for including comments from .proto file (#85, #645)
nanopb-0.4.4 (2020-11-25)
Fix memory leak with oneofs and PB_ENABLE_MALLOC (#615, GHSA-85rr-4rh9-hhwh)
Fix generator error when oneof contains a single unresolved field size (#610)
Fix unsigned enums not working correctly inside OneOf (#611)
Fix recursion depth exceeded error in generator (#493)
Add '--version' option to nanopb_generator.py (#607)
Add support for proto3 optional fields introduced in protoc 3.12 (#591)
Add better error message when enum type is not found (#592)
Allow specifying descriptorsize on field level also (#546)
Allow multiple targets to be created by calling NANOPB_GENERATE_CPP() (#596)
Conanfile: Add protobuf-nanopb library to cpp_info.libs (#605)
Include version number in generator verbose output
Convert documentation to markdown format (#587)
Remove outdated transitional generator/nanopb/options.proto.
Test case improvements
Documentation improvements
nanopb-0.4.3 (2020-09-21)
Fix generator error when output is in current directory (#553)
Fix issue with unknown message sizes being referred inside oneof encoded size calculation (#569)
Fix problem with [default=xxxx, (nanopb).proto3=true] field option combination (#558)
Fix cross compilation with Conan build system (#568)
Better support C++ types in generated structs (#577)
CMake rule improvements (#554, #555, #556, #561, #564)
Generator: fix compatibility bug with Python 3.1 and earlier
Make Linux and Mac packages use grpcio protoc
Make pb_decode_varint32() reject overlong varint encodings.
Performance optimizations
Test case improvements
Documentation improvements
NOTE: version 0.4.3 changes layout of pb_msgdesc_t. It requires recompiling .pb.c files and
thus breaks ABI compatibility. In general, ABI compatibility is not currently maintained
between any nanopb versions.
NOTE: There was an apparent false positive virus identification by Windows Defender of the
PyInstaller 3.6 based Windows package nanopb-0.4.3-windows-x86.zip. The package was replaced
by nanopb-0.4.3-p1-windows-x86.zip with rebuilt PyInstaller 4.0, which seems to avoid the problem.
Actual nanopb code is unchanged between the packages.
nanopb-0.4.2 (2020-06-23)
Fix buffer overflow when encoding bytes with size set to 65535 (#547, GHSA-3p39-mfxg-hrq4)
Fix segfault with pointer fields and proto3_singular_msgs = true. (#504,#505)
Fix Windows 10 temp file handling (#486)
Fix macro name conflicts (ATMEGA32U4 UENUM and more) (#522)
Fix generator error with nested messages and default values (#487)
Fix nanopb_generator exception on enums with aliases (#535)
Fix compile error when struct members are called X or a (#492)
Fix sizeof(union ...) fallback not compiling with C++ (#415, #494)
Fix "missing required field" error with submessage callback (#544)
Fix field descriptor sizing with submsg_callback option (#545)
Fix protoc calling on Python 2 (#503)
Fix handling of varying NaN representations in PB_CONVERT_DOUBLE_FLOAT (#543)
Fix clang undefined behavior sanitizer errors.
Change generator to use Python 3 by default (#441, #509)
Binary packages updated to use Python 3 and grpcio-tools
Add support for infinity and nan floating-point defaults (#530, #538)
Add generator option sort_by_tag (#542)
Add type_override option to override type defined in .proto (#497)
Set proto2 enum fields to first value when no default is given, even if nonzero (#532,#539)
Include protoc-gen-nanopb in path in protoc wrapper script
Properly pass error status from protoc wrapper when calling binary protoc
Generator: pass --include_imports when calling protoc (#494)
Create intermediate directories when writing files to relative path (#512)
Add current directory to include path for protoc (#499)
Update readme to use nanopb_generator.py directly
Regression test for proto3 incorrectly considered empty (#504)
CMake: change package name to Nanopb for cmake 3.17 compatibility (#506)
CMake: remove find_package(PythonInterp) (#508)
CMake: use split --nanopb_opt only on protoc >= 3.6 (#515)
CMake: Relax python version spec, allowing Python3. (#534)
Swift package manager (#549)
Rename BUILD as BUILD.bazel (#537)
Note: Windows binary packages in 0.4.2 and later require Windows 7 or newer.
nanopb-0.4.1 (2020-02-02)
Fix invalid free() after failed realloc() (GHSA-gcx3-7m76-287p)
Avoid overflows in allocation for packed fields.
Verify stream size before allocating string / bytes.
Add workaround for avr-libc realloc() bug (#475)
Fix bug with field numbers >255 (#407)
Fix compilation error on platforms without uint8_t (#485)
Fix warnings on Python3.8 (#399, #467)
Make fixed_count option work when combined with FT_POINTER.
Add missing #define for submsg callbacks, add regression test (#472)
Fix ImportError when using generator/protoc with Python 3
Remove accidental debug code in generator
Reduce stack usage (#484)
Remove PB_FIELDINFO_WIDTH option (#473)
Add nanopb-specific package name option (#422)
Add testcase for Any type (#163)
Add exclude option also from .proto/.options
Set default include path in the grpc_tools protoc wrapper.
Add workaround for python-protobuf 3.6.1 bug (#478)
Detect invalid wire type when decoding fields.
Improved fuzz testing
nanopb-0.4.0 (2019-12-20)
New field descriptor format.
Make nanopb_generator automatically compile .proto files (#462)
Allow installing as Python package from pip (#460)
Use protoc from grpcio-tools Python package if available (#463)
Change proto3 message types to be optional (#308, #452)
Add pb_decode_ex(), pb_encode_ex() functions.
Automatically rebuild nanopb_pb2.py
Use plugin.proto that comes with python-protobuf (#234)
Allow specifying a per-message callback. (#175)
Improve callback handling inside oneofs. (#175)
Introduce new compile time flag: PB_VALIDATE_UTF8 (#437)
Add TypenameMangling.M_PACKAGE_INITIALS (#394)
Introduce new compile time flag: PB_ENCODE_ARRAYS_UNPACKED (#427)
Add default_has option (#423)
Add option for including extra files from .pb.h
Add generator option to error out on unmatched options (#458)
Generator: Allow comma separated options in plugin mode (#343)
Allow comma-separated option parsing to handle `#include` (#450)
Remove timestamp from generated files by default, add -t to keep it.
Make --no-strip-path default (#326)
Fix .options file case sensitivity on Windows.
Fix generator error with mangle_names option (#380)
Take int_size setting into account in calculating message sizes (#373)
.gitignore: don't ignore generator-bin files (#419)
Cleanup .pb.h header format
Make tests run on AVR and STM32
Add PB_CONVERT_DOUBLE_FLOAT setting to convert doubles on AVR.
Store field descriptor constants in flash on AVR (#464)
Added "f" suffix to const float declarations. (#453)
Fix clang-tidy warnings about using signed integers in binary bitwise operations (#451)
Add C++ message descriptors helper (#384)
Implement conan recipe (#378)
CMake: Split nanopb_out command (#454)
CMake: install created shared library(dll) in windows to the binary folder (#447)
nanopb-0.3.9.9 (2022-04-23)
Fix Xcode analyzer warnings (#667, #674)
Fix clang sanitizer warnings
Note: there are no known functional differences between 0.3.9.8 and 0.3.9.9.
The changes are merely to fix warnings introduced by new compiler versions.
nanopb-0.3.9.8 (2021-03-22)
Fix invalid free() with oneof (#647, GHSA-7mv5-5mxh-qg88)
Don't generate lines with trailing spaces (#622)
Verify stream size before allocating string / bytes (#620)
nanopb-0.3.9.7 (2020-11-25)
Fix memory leak with oneofs and PB_ENABLE_MALLOC (#615, GHSA-85rr-4rh9-hhwh)
Fix unsigned enums not working correctly inside OneOf (#611)
Add '--version' option to nanopb_generator.py (#607)
SwiftPM rule updates (#567, #585)
nanopb-0.3.9.6 (2020-06-23)
Fix buffer overflow when encoding bytes with size set to 65535 (#547, GHSA-3p39-mfxg-hrq4)
Fix proto3 submessage improperly considered empty (#504)
Fix ImportError when using generator/protoc with Python 3
Add build rules for Swift package manager (#549)
nanopb-0.3.9.5 (2020-02-02)
Fix invalid free() after failed realloc() (GHSA-gcx3-7m76-287p)
Add workaround for avr-libc realloc() bug (#475)
Fix empty submessages getting encoded in proto3 mode (#395)
Avoid overflows in allocation for packed fields.
nanopb-0.3.9.4 (2019-10-13)
Fix undefined behavior with bool fields (#434)
Fix enum min/max defines when values are not in order (#405)
Fix network_server socket example with zero-length strings (#421)
Don't call stream read callback with count=0 (#421)
Add compile time flag PB_ENCODE_ARRAYS_UNPACKED (#427)
nanopb-0.3.9.3 (2019-03-08)
NOTE: nanopb-0.3.9.3.tar.gz before 2019-03-21 was accidentally from 0.4 branch (#388)
Fix fixed size and callback repeated fields inside proto3 submessages (#376, #382, #386)
Fix incorrect PB_STATIC_ASSERT for bytes inside oneof (#363)
Fix generator error with mangle_names option (#380)
Generator: Allow comma separated options in plugin mode (#343)
nanopb-0.3.9.2 (2018-11-10)
Erroneous free() when using callbacks combined with PB_ENABLE_MALLOC (#346)
Fix possible null-pointer dereference in decode_callback_field (#342)
Fix FindNanopb.cmake on Windows (#335)
Fix large generator memory usage with oneof fields (#338)
Fix error in splint test (#359)
Allow cmake to build as a shared library (#352, #353)
Add --no-strip-path command line option (#326)
Option for flattening nested protobuf names (#333)
Documentation fixes (#329, #350, #358)
Better error messages (#351)
nanopb-0.3.9.1 (2018-04-14)
Fix handling of special characters in string/bytes default values (issue #322)
Fix encoding of negative numbers with PB_WITHOUT_64BIT (#285)
Fix _zero initializer for enums that don't begin at 0. (#295)
Multiple CMake fixes (#296, #299, #304, #312, #320)
Fix compiler warnings (#305)
Fix scons rules for Python 3
Add check for large extension field number (issue #306)
Updated included descriptor.proto version (#314)
Resolve oneof sizes symbolically when needed (#311)
Add fixed_count option (#260)
Add some verbose prints in generator (issue #238)
Add test/example of using 'map' type. (#289)
nanopb-0.3.9 (2017-09-23)
Fix bugs in proto3 encoding of submessages (#256)
Fix message size calculation for arrays of size 1 (#253)
Fix segfault with FT_CALLBACK inside FT_POINTER (#259)
Properly detect truncated tags in corrupted messages (#277)
Make pb_decode_varint32 overflow checks exact (#258)
Add option to build without 64-bit support (#86)
Add options to define source and header file extensions (#264)
Add pb_en/decode_nullterminated() (part of #278)
Add pb_decode_delimited_noinit (#284)
CMake: add dependency for .options file (#265)
CMake: change use of relative paths (#250,#271,#273)
Better error message for missing max_size option (#281)
Travis-CI build fixes (#283)
Add Bazel build system file (#266)
nanopb-0.3.8 (2017-03-05)
Fix problems with multiple oneofs in same message (#229)
Zero-valued extension fields were mistakenly ignored by encoder (#242)
Multiple fixes related to proto3 mode (#242, #245, #247, #249)
Fix potential unaligned access (#226, #227)
Fix documentation for protoc --plugin argument (#239)
Extend inline / fixed length bytes array support (#244)
Add new option max_length for strings (#107)
Make string substream API more robust (#230)
Make pb_decode_varint32 public API (#231)
Allow overriding proto3 mode (#228)
Add optional enum->string mapping function (#223)
Add transitional options.proto file (#241)
Add better error message on Python library version incompatibility (#240)
Include version number in PlatformIO library.json (#222)
CMake build script changes (#236, #237)
Change download links to https
Improvements to test cases.
nanopb-0.3.7 (2016-10-30)
Add support for proto3-style singular fields (#182, #206, #216)
Updated binary package protoc to version 3.1.0
Add FT_INLINE allocation of bytes fields (#211)
Include package name in include guard (#207)
Fix missing warning with large bytes fields (issue #220)
Added CMake project (#208)
Add bazel BUILD file for nanopb (#209)
Added an AUTHORS file (#211)
Documentation updates
Improvements to test cases.
nanopb-0.3.6 (2016-06-19)
Protect against corrupted _count fields in pb_release (#205)
Fix error in STATIC_ASSERT with multiple files (#203)
Add -D option to specify output directory (#193)
Generate MIN/MAX/ARRAYSIZE defines for enums (#194)
Generate comments about uncalculable message sizes (#195)
Documentation updates (#196, #201)
Improvements to test cases.
nanopb-0.3.5 (2016-02-13)
NOTE: If you are using pb_syshdr.h, you will need to add uint_least8_t
definition. See docs/migration.rst for details.
Fix generator crash with Enum inside Oneof (#188)
Fix some generator regressions related to .options file path (#172)
Add support for platforms without uint8_t (#191)
Allow const parameter to pb_istream_from_buffer (#152)
Ignore null pointers in pb_release() (#183)
Add support for anonymous unions (#184)
Add Python3 support to the generator (#169)
Add code generator insertion points to generated files (#178)
Improvements to CMake script (#181)
Improvements to test cases.
nanopb-0.3.4 (2015-09-26)
Fix handling of unsigned 8- and 16-bit enums (issue 164)
Fix generator on systems where python = python3. (issue 155)
Fix compiler warning on GCC 5.x (issue 171)
Make the generator better handle imported .protos (issue 165)
Add packed_enum option to generator.
Add syntax= line to .proto files (issue 167)
Add PlatformIO registry manifest file. (pr 156)
nanopb-0.3.3 (2015-04-10)
Fix missing files in Linux binary package (issue 146)
Fix generator bug when oneof is first field in a message. (issue 142)
Fix generator error when long_names:false is combined with Oneofs. (issue 147)
Fix oneof submessage initialization bug. (issue 149)
Fix problem with plugin options on Python 2.7.2 and older. (issue 153)
Fix crash when callback is inside oneof field. (issue 148)
Switch to .tar.gz format for Mac OS X packages. (issue 154)
Always define enum long names so that cross-file references work. (issue 118)
Add msgid generator option. (issue 151)
Improve comment support in .options files. (issue 145)
Updates for the CMake rule file, add cmake example.
Better error messages for syntax errors in .options file
nanopb-0.3.2 (2015-01-24)
Fix memory leaks with PB_ENABLE_MALLOC with some submessage hierarchies (issue 138)
Implement support for oneofs (C unions). (issues 131, 141)
Add int_size option for generator (issue 139)
Add compilation option to disable struct packing. (issue 136)
Change PB_RETURN_ERROR() macro to avoid compiler warnings (issue 140)
Fix build problems with protoc 3.0.0
Add support for POINTER type in extensions
Initialize also extension fields to defaults in pb_decode().
Detect too large varint values when decoding.
nanopb-0.3.1 (2014-09-11)
Fix security issue due to size_t overflows. (issue 132)
Fix memory leak with duplicated fields and PB_ENABLE_MALLOC
Fix crash if pb_release() is called twice.
Fix cyclic message support (issue 130)
Fix error in generated initializers for repeated pointer fields.
Improve tests (issues 113, 126)
nanopb-0.3.0 (2014-08-26)
NOTE: See docs/migration.html or online at
http://koti.kapsi.fi/~jpa/nanopb/docs/migration.html
for changes in this version. Most importantly, you need to add
pb_common.c to the list of files to compile.
Separated field iterator logic to pb_common.c (issue 128)
Change the _count fields to use pb_size_t datatype (issue 82)
Added PB_ prefix to macro names (issue 106)
Added #if version guard to generated files (issue 129)
Added migration document
nanopb-0.2.9.5 (2020-06-23)
Fix buffer overflow when encoding bytes with size set to 65535 (#547, GHSA-3p39-mfxg-hrq4)
Backport Python 3 and protoc 3.x fixes to test cases
nanopb-0.2.9.4 (2020-02-02)
Fix invalid free() after failed realloc() (GHSA-gcx3-7m76-287p)
Add workaround for avr-libc realloc() bug (#475)
nanopb-0.2.9.3 (2016-06-19)
Protect against corrupted _count fields in pb_release (#205)
nanopb-0.2.9.2 (2015-01-24)
Fix memory leaks with PB_ENABLE_MALLOC with some submessage hierarchies (issue 138)
Fix compilation error with generated initializers for repeated pointer fields
nanopb-0.2.9.1 (2014-09-11)
Fix security issue due to size_t overflows. (issue 132)
Fix memory leak with duplicated fields and PB_ENABLE_MALLOC
Fix crash if pb_release() is called twice.
nanopb-0.2.9 (2014-08-09)
NOTE: If you are using the -e option with the generator, you have
to prepend . to the argument to get the same behaviour as before.
Do not automatically add a dot with generator -e option. (issue 122)
Fix problem with .options file and extension fields. (issue 125)
Don't use SIZE_MAX macro, as it is not in C89. (issue 120)
Generate #defines for initializing message structures. (issue 79)
Add skip_message option to generator. (issue 121)
Add PB_PACKED_STRUCT support for Keil MDK-ARM toolchain (issue 119)
Give better messages about the .options file path. (issue 124)
Improved tests
nanopb-0.2.8 (2014-05-20)
Fix security issue with PB_ENABLE_MALLOC. (issue 117)
Add option to not add timestamps to .pb.h and .pb.c preambles. (issue 115)
Documentation updates
Improved tests
nanopb-0.2.7 (2014-04-07)
Fix bug with default values for extension fields (issue 111)
Fix some MISRA-C warnings (issue 91)
Implemented optional malloc() support (issue 80)
Changed pointer-type bytes field datatype
Add a "found" field to pb_extension_t (issue 112)
Add convenience function pb_get_encoded_size() (issue 16)
nanopb-0.2.6 (2014-02-15)
Fix generator error with bytes callback fields (issue 99)
Fix warnings about large integer constants (issue 102)
Add comments to where STATIC_ASSERT is used (issue 96)
Add warning about unknown field names on .options (issue 105)
Move descriptor.proto to google/protobuf subdirectory (issue 104)
Improved tests
nanopb-0.2.5 (2014-01-01)
Fix a bug with encoding negative values in int32 fields (issue 97)
Create binary packages of the generator + dependencies (issue 47)
Add support for pointer-type fields to the encoder (part of issue 80)
Fixed path in FindNanopb.cmake (issue 94)
Improved tests
nanopb-0.2.4 (2013-11-07)
Remove the deprecated NANOPB_INTERNALS functions from public API.
Document the security model.
Check array and bytes max sizes when encoding (issue 90)
Add #defines for maximum encoded message size (issue 89)
Add #define tags for extension fields (issue 93)
Fix MISRA C violations (issue 91)
Clean up pb_field_t definition with typedefs.
nanopb-0.2.3 (2013-09-18)
Improve compatibility by removing ternary operator from initializations (issue 88)
Fix build error on Visual C++ (issue 84, patch by Markus Schwarzenberg)
Don't stop on unsupported extension fields (issue 83)
Add an example pb_syshdr.h file for non-C99 compilers
Reorganize tests and examples into subfolders (issue 63)
Switch from Makefiles to scons for building the tests
Make the tests buildable on Windows
nanopb-0.2.2 (2013-08-18)
Add support for extension fields (issue 17)
Fix unknown fields in empty message (issue 78)
Include the field tags in the generated .pb.h file.
Add pb_decode_delimited and pb_encode_delimited wrapper functions (issue 74)
Add a section in top of pb.h for changing compilation settings (issue 76)
Documentation improvements (issues 12, 77 and others)
Improved tests
nanopb-0.2.1 (2013-04-14)
NOTE: The default callback function signature has changed.
If you don't want to update your code, define PB_OLD_CALLBACK_STYLE.
Change the callback function to use void** (issue 69)
Add support for defining the nanopb options in a separate file (issue 12)
Add support for packed structs in IAR and MSVC (in addition to GCC) (issue 66)
Implement error message support for the encoder side (issue 7)
Handle unterminated strings when encoding (issue 68)
Fix bug with empty strings in repeated string callbacks (issue 73)
Fix regression in 0.2.0 with optional callback fields (issue 70)
Fix bugs with empty message types (issues 64, 65)
Fix some compiler warnings on clang (issue 67)
Some portability improvements (issues 60, 62)
Various new generator options
Improved tests
nanopb-0.2.0 (2013-03-02)
NOTE: This release requires you to regenerate all .pb.c
files. Files generated by older versions will not
compile anymore.
Reformat generated .pb.c files using macros (issue 58)
Rename PB_HTYPE_ARRAY -> PB_HTYPE_REPEATED
Separate PB_HTYPE to PB_ATYPE and PB_HTYPE
Move STATIC_ASSERTs to .pb.c file
Added CMake file (by Pavel Ilin)
Add option to give file extension to generator (by Michael Haberler)
Documentation updates
nanopb-0.1.9 (2013-02-13)
Fixed error message bugs (issues 52, 56)
Sanitize #ifndef filename (issue 50)
Performance improvements
Add compile-time option PB_BUFFER_ONLY
Add Java package name to nanopb.proto
Check for sizeof(double) == 8 (issue 54)
Added generator option to ignore some fields. (issue 51)
Added generator option to make message structs packed. (issue 49)
Add more test cases.
nanopb-0.1.8 (2012-12-13)
Fix bugs in the enum short names introduced in 0.1.7 (issues 42, 43)
Fix STATIC_ASSERT macro when using multiple .proto files. (issue 41)
Fix missing initialization of istream.errmsg
Make tests/Makefile work for non-gcc compilers (issue 40)
nanopb-0.1.7 (2012-11-11)
Remove "skip" mode from pb_istream_t callbacks. Example implementation had a bug. (issue 37)
Add option to use shorter names for enum values (issue 38)
Improve options support in generator (issues 12, 30)
Add nanopb version number to generated files (issue 36)
Add extern "C" to generated headers (issue 35)
Add names for structs to allow forward declaration (issue 39)
Add buffer size check in example (issue 34)
Fix build warnings on MS compilers (issue 33)
nanopb-0.1.6 (2012-09-02)
Reorganize the field decoder interface (issue 2)
Improve performance in submessage decoding (issue 28)
Implement error messages in the decoder side (issue 7)
Extended testcases (alltypes test is now complete).
Fix some compiler warnings (issues 25, 26, 27, 32).
nanopb-0.1.5 (2012-08-04)
Fix bug in decoder with packed arrays (issue 23).
Extended testcases.
Fix some compiler warnings.
nanopb-0.1.4 (2012-07-05)
Add compile-time options for easy-to-use >255 field support.
Improve the detection of missing required fields.
Added example on how to handle union messages.
Fix generator error with .proto without messages.
Fix problems that stopped the code from compiling with some compilers.
Fix some compiler warnings.
nanopb-0.1.3 (2012-06-12)
Refactor the field encoder interface.
Improve generator error messages (issue 5)
Add descriptor.proto into the #include exclusion list
Fix some compiler warnings.
nanopb-0.1.2 (2012-02-15)
Make the generator to generate include for other .proto files (issue 4).
Fixed generator not working on Windows (issue 3)
nanopb-0.1.1 (2012-01-14)
Fixed bug in encoder with 'bytes' fields (issue 1).
Fixed a bug in the generator that caused a compiler error on sfixed32 and sfixed64 fields.
Extended testcases.
nanopb-0.1.0 (2012-01-06)
First stable release.

7
lib/nanopb/CMakeLists.txt Normal file
View File

@@ -0,0 +1,7 @@
add_library(nanopb
pb_common.c
pb_decode.c
pb_encode.c
)
target_include_directories(nanopb INTERFACE .)

144
lib/nanopb/CMakeLists.txt.original Normal file
View File

@@ -0,0 +1,144 @@
cmake_minimum_required(VERSION 2.8.12)
project(nanopb C)
set(nanopb_VERSION_STRING nanopb-0.4.8-dev)
set(nanopb_SOVERSION 0)
string(REPLACE "nanopb-" "" nanopb_VERSION ${nanopb_VERSION_STRING})
option(BUILD_SHARED_LIBS "Build shared libraries" OFF)
option(BUILD_STATIC_LIBS "Build static libraries" ON)
option(nanopb_BUILD_RUNTIME "Build the headers and libraries needed at runtime" ON)
option(nanopb_BUILD_GENERATOR "Build the protoc plugin for code generation" ON)
option(nanopb_MSVC_STATIC_RUNTIME "Link static runtime libraries" ON)
find_program(nanopb_PROTOC_PATH protoc HINTS generator-bin generator)
if(NOT EXISTS ${nanopb_PROTOC_PATH})
message(FATAL_ERROR "protoc compiler not found")
endif()
if(NOT DEFINED CMAKE_DEBUG_POSTFIX)
set(CMAKE_DEBUG_POSTFIX "d")
endif()
include(GNUInstallDirs)
if(MSVC AND nanopb_MSVC_STATIC_RUNTIME)
foreach(flag_var
CMAKE_C_FLAGS CMAKE_C_FLAGS_DEBUG CMAKE_C_FLAGS_RELEASE
CMAKE_C_FLAGS_MINSIZEREL CMAKE_C_FLAGS_RELWITHDEBINFO)
if(${flag_var} MATCHES "/MD")
string(REGEX REPLACE "/MD" "/MT" ${flag_var} "${${flag_var}}")
endif(${flag_var} MATCHES "/MD")
endforeach(flag_var)
endif()
if(NOT DEFINED CMAKE_INSTALL_CMAKEDIR)
set(CMAKE_INSTALL_CMAKEDIR "${CMAKE_INSTALL_LIBDIR}/cmake/nanopb")
endif()
find_package(Python REQUIRED COMPONENTS Interpreter)
execute_process(
COMMAND ${Python_EXECUTABLE} -c
"import os.path, sys, sysconfig; print(os.path.relpath(sysconfig.get_path('purelib'), start=sys.prefix))"
OUTPUT_VARIABLE PYTHON_INSTDIR
OUTPUT_STRIP_TRAILING_WHITESPACE
)
if(nanopb_BUILD_GENERATOR)
set(generator_protos nanopb)
foreach(generator_proto IN LISTS generator_protos)
string(REGEX REPLACE "([^;]+)" "${PROJECT_SOURCE_DIR}/generator/proto/\\1.proto" generator_proto_file "${generator_proto}")
string(REGEX REPLACE "([^;]+)" "\\1_pb2.py" generator_proto_py_file "${generator_proto}")
add_custom_command(
OUTPUT ${generator_proto_py_file}
COMMAND ${nanopb_PROTOC_PATH} --python_out=${PROJECT_BINARY_DIR} -I${PROJECT_SOURCE_DIR}/generator/proto ${generator_proto_file}
DEPENDS ${generator_proto_file}
)
add_custom_target("generate_${generator_proto_py_file}" ALL DEPENDS ${generator_proto_py_file})
install(
FILES ${PROJECT_BINARY_DIR}/${generator_proto_py_file}
${generator_proto_file}
DESTINATION ${PYTHON_INSTDIR}/proto/
)
endforeach()
install( FILES generator/proto/_utils.py
generator/proto/__init__.py
DESTINATION ${PYTHON_INSTDIR}/proto/ )
endif()
if(WIN32)
install(
PROGRAMS
generator/nanopb_generator.py
generator/protoc-gen-nanopb.bat
DESTINATION ${CMAKE_INSTALL_BINDIR}
)
else()
install(
PROGRAMS
generator/nanopb_generator.py
generator/protoc-gen-nanopb
DESTINATION ${CMAKE_INSTALL_BINDIR}
)
endif()
if(nanopb_BUILD_RUNTIME)
if(BUILD_SHARED_LIBS)
add_library(protobuf-nanopb SHARED
pb.h
pb_common.h
pb_common.c
pb_encode.h
pb_encode.c
pb_decode.h
pb_decode.c)
set_target_properties(protobuf-nanopb PROPERTIES
SOVERSION ${nanopb_SOVERSION})
install(TARGETS protobuf-nanopb EXPORT nanopb-targets
ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR}
LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}
RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR})
target_include_directories(protobuf-nanopb INTERFACE
$<INSTALL_INTERFACE:${CMAKE_INSTALL_INCLUDEDIR}>
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}>
)
endif()
if(BUILD_STATIC_LIBS)
add_library(protobuf-nanopb-static STATIC
pb.h
pb_common.h
pb_common.c
pb_encode.h
pb_encode.c
pb_decode.h
pb_decode.c)
set_target_properties(protobuf-nanopb-static PROPERTIES
OUTPUT_NAME protobuf-nanopb)
install(TARGETS protobuf-nanopb-static EXPORT nanopb-targets
ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR})
target_include_directories(protobuf-nanopb-static INTERFACE
$<INSTALL_INTERFACE:${CMAKE_INSTALL_INCLUDEDIR}>
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}>
)
endif()
configure_file(extra/nanopb-config-version.cmake.in
nanopb-config-version.cmake @ONLY)
install(EXPORT nanopb-targets
DESTINATION ${CMAKE_INSTALL_CMAKEDIR}
NAMESPACE nanopb::)
install(FILES extra/nanopb-config.cmake
${CMAKE_CURRENT_BINARY_DIR}/nanopb-config-version.cmake
DESTINATION ${CMAKE_INSTALL_CMAKEDIR})
install(FILES pb.h pb_common.h pb_encode.h pb_decode.h
DESTINATION ${CMAKE_INSTALL_INCLUDEDIR})
endif()

32
lib/nanopb/CONTRIBUTING.md Normal file
View File

@@ -0,0 +1,32 @@
Contributing to Nanopb development
==================================
Reporting issues and requesting features
----------------------------------------
Feel free to report any issues you see or features you would like
to see in the future to the Github issue tracker. Using the templates
below is preferred:
* [Report a bug](https://github.com/nanopb/nanopb/issues/new?body=**Steps%20to%20reproduce%20the%20issue**%0a%0a1.%0a2.%0a3.%0a%0a**What%20happens?**%0A%0A**What%20should%20happen?**&labels=Type-Defect)
* [Request a feature](https://github.com/nanopb/nanopb/issues/new?body=**What%20should%20the%20feature%20do?**%0A%0A**In%20what%20situation%20would%20the%20feature%20be%20useful?**&labels=Type-Enhancement)
Requesting help
---------------
If there is something strange going on, but you do not know if
it is actually a bug in nanopb, try asking first on the
[discussion forum](https://groups.google.com/forum/#!forum/nanopb).
Pull requests
-------------
Pull requests are welcome!
If it is not obvious from the commit message, please indicate the
same information as you would for an issue report:
* What functionality it fixes/adds.
* How can the problem be reproduced / when would the feature be useful.

20
lib/nanopb/LICENSE.txt Normal file
View File

@@ -0,0 +1,20 @@
Copyright (c) 2011 Petteri Aimonen <jpa at nanopb.mail.kapsi.fi>
This software is provided 'as-is', without any express or
implied warranty. In no event will the authors be held liable
for any damages arising from the use of this software.
Permission is granted to anyone to use this software for any
purpose, including commercial applications, and to alter it and
redistribute it freely, subject to the following restrictions:
1. The origin of this software must not be misrepresented; you
must not claim that you wrote the original software. If you use
this software in a product, an acknowledgment in the product
documentation would be appreciated but is not required.
2. Altered source versions must be plainly marked as such, and
must not be misrepresented as being the original software.
3. This notice may not be removed or altered from any source
distribution.

63
lib/nanopb/Package.swift Normal file
View File

@@ -0,0 +1,63 @@
// swift-tools-version:5.0
// The swift-tools-version declares the minimum version of Swift required to build this package.
import PackageDescription
let package = Package(
name: "nanopb",
products: [
.library(
name: "nanopb",
targets: ["nanopb"]
)
],
targets: [
.target(
name: "nanopb",
path: ".",
sources: [
"pb.h",
"pb_common.h",
"pb_common.c",
"pb_decode.h",
"pb_decode.c",
"pb_encode.h",
"pb_encode.c"
],
publicHeadersPath: "spm_headers",
cSettings: [
.define("PB_FIELD_32BIT", to: "1"),
.define("PB_NO_PACKED_STRUCTS", to: "1"),
.define("PB_ENABLE_MALLOC", to: "1"),
]
),
.testTarget(
name: "swift-test",
dependencies: [
"nanopb",
],
path: "spm-test/swift",
cSettings: [
.headerSearchPath("../"),
.define("PB_FIELD_32BIT", to: "1"),
.define("PB_NO_PACKED_STRUCTS", to: "1"),
.define("PB_ENABLE_MALLOC", to: "1"),
]
),
.testTarget(
name: "objc-test",
dependencies: [
"nanopb",
],
path: "spm-test/objc",
cSettings: [
.headerSearchPath("../"),
.define("PB_FIELD_32BIT", to: "1"),
.define("PB_NO_PACKED_STRUCTS", to: "1"),
.define("PB_ENABLE_MALLOC", to: "1"),
]
)
]
)

99
lib/nanopb/README.md Normal file
View File

@@ -0,0 +1,99 @@
Nanopb - Protocol Buffers for Embedded Systems
==============================================
![Latest change](https://github.com/nanopb/nanopb/actions/workflows/trigger_on_code_change.yml/badge.svg)
![Weekly build](https://github.com/nanopb/nanopb/actions/workflows/trigger_on_schedule.yml/badge.svg)
Nanopb is a small code-size Protocol Buffers implementation in ansi C. It is
especially suitable for use in microcontrollers, but fits any memory
restricted system.
* **Homepage:** https://jpa.kapsi.fi/nanopb/
* **Git repository:** https://github.com/nanopb/nanopb/
* **Documentation:** https://jpa.kapsi.fi/nanopb/docs/
* **Forum:** https://groups.google.com/forum/#!forum/nanopb
* **Stable version downloads:** https://jpa.kapsi.fi/nanopb/download/
* **Pre-release binary packages:** https://github.com/nanopb/nanopb/actions/workflows/binary_packages.yml
Using the nanopb library
------------------------
To use the nanopb library, you need to do two things:
1. Compile your .proto files for nanopb, using `protoc`.
2. Include *pb_encode.c*, *pb_decode.c* and *pb_common.c* in your project.
The easiest way to get started is to study the project in "examples/simple".
It contains a Makefile, which should work directly under most Linux systems.
However, for any other kind of build system, see the manual steps in
README.txt in that folder.
Generating the headers
----------------------
Protocol Buffers messages are defined in a `.proto` file, which follows a standard
format that is compatible with all Protocol Buffers libraries. To use it with nanopb,
you need to generate `.pb.c` and `.pb.h` files from it:
python generator/nanopb_generator.py myprotocol.proto # For source checkout
generator-bin/nanopb_generator myprotocol.proto # For binary package
(Note: For instructions for nanopb-0.3.9.x and older, see the documentation
of that particular version [here](https://github.com/nanopb/nanopb/blob/maintenance_0.3/README.md))
The binary packages for Windows, Linux and Mac OS X should contain all necessary
dependencies, including Python, python-protobuf library and protoc. If you are
using a git checkout or a plain source distribution, you will need to install
Python separately. Once you have Python, you can install the other dependencies
with `pip install --upgrade protobuf grpcio-tools`.
You can further customize the header generation by creating an `.options` file.
See [documentation](https://jpa.kapsi.fi/nanopb/docs/concepts.html#modifying-generator-behaviour) for details.
Running the tests
-----------------
If you want to perform further development of the nanopb core, or to verify
its functionality using your compiler and platform, you'll want to run the
test suite. The build rules for the test suite are implemented using Scons,
so you need to have that installed (ex: `sudo apt install scons` or `pip install scons`).
To run the tests:
cd tests
scons
This will show the progress of various test cases. If the output does not
end in an error, the test cases were successful.
Note: Mac OS X by default aliases 'clang' as 'gcc', while not actually
supporting the same command line options as gcc does. To run tests on
Mac OS X, use: `scons CC=clang CXX=clang`. Same way can be used to run
tests with different compilers on any platform.
For embedded platforms, there is currently support for running the tests
on STM32 discovery board and [simavr](https://github.com/buserror/simavr)
AVR simulator. Use `scons PLATFORM=STM32` and `scons PLATFORM=AVR` to run
these tests.
Build systems and integration
-----------------------------
Nanopb C code itself is designed to be portable and easy to build
on any platform. Often the bigger hurdle is running the generator which
takes in the `.proto` files and outputs `.pb.c` definitions.
There exist build rules for several systems:
* **Makefiles**: `extra/nanopb.mk`, see `examples/simple`
* **CMake**: `extra/FindNanopb.cmake`, see `examples/cmake`
* **SCons**: `tests/site_scons` (generator only)
* **Bazel**: `BUILD` in source root
* **Conan**: `conanfile.py` in source root
* **PlatformIO**: https://platformio.org/lib/show/431/Nanopb
* **PyPI/pip**: https://pypi.org/project/nanopb/
* **vcpkg**: https://vcpkg.info/port/nanopb
And also integration to platform interfaces:
* **Arduino**: http://platformio.org/lib/show/1385/nanopb-arduino

22
lib/nanopb/WORKSPACE Normal file
View File

@@ -0,0 +1,22 @@
workspace(name = "com_github_nanopb_nanopb")
load("//extra/bazel:nanopb_deps.bzl", "nanopb_deps")
nanopb_deps()
load("@rules_python//python:repositories.bzl", "python_register_toolchains")
python_register_toolchains(
name = "python3_9",
python_version = "3.9",
)
load("//extra/bazel:python_deps.bzl", "nanopb_python_deps")
load("@python3_9//:defs.bzl", "interpreter")
nanopb_python_deps(interpreter)
load("//extra/bazel:nanopb_workspace.bzl", "nanopb_workspace")
nanopb_workspace()

6
lib/nanopb/build.py Normal file
View File

@@ -0,0 +1,6 @@
from conan.packager import ConanMultiPackager
if __name__ == "__main__":
builder = ConanMultiPackager(build_policy="outdated")
builder.add_common_builds(shared_option_name=None)
builder.run()

7
lib/nanopb/conan-wrapper/CMakeLists.txt Normal file
View File

@@ -0,0 +1,7 @@
cmake_minimum_required(VERSION 2.8.12)
project(cmake_wrapper)
include(${CMAKE_CURRENT_BINARY_DIR}/conanbuildinfo.cmake)
conan_basic_setup()
add_subdirectory(".." "nanopb")

33
lib/nanopb/conanfile.py Normal file
View File

@@ -0,0 +1,33 @@
from conans import ConanFile, CMake, tools
from os import path
class NanoPbConan(ConanFile):
name = "nanopb"
version = "0.4.8-dev"
license = "zlib"
url = "https://jpa.kapsi.fi/nanopb/"
description = "Protocol Buffers with small code size"
settings = "os_build", "compiler", "build_type", "arch"
generators = "cmake"
exports = '*'
options = {
"fPIC": [True, False],
}
default_options = {
"fPIC": True,
}
def configure(self):
if self.settings.os_build == "Windows" and self.settings.compiler == "Visual Studio":
del self.options.fPIC
def build(self):
cmake = CMake(self)
cmake.configure(source_folder=path.join(self.source_folder, "conan-wrapper"))
cmake.build()
cmake.install()
def package_info(self):
self.cpp_info.includedirs = ["include"]
self.cpp_info.libdirs = ["lib"]
self.cpp_info.libs = ["protobuf-nanopb"]

23
lib/nanopb/docs/Makefile Normal file
View File

@@ -0,0 +1,23 @@
INPUTS = index.md concepts.md reference.md security.md migration.md whats_new.md
all: $(INPUTS:.md=.html)
tmp_menu.html: $(INPUTS)
echo '<div id="index">' > $@
(echo '<h2>Documentation index</h2>'; \
for file in $^; do echo -n '1. ['; sed -n '1 s!^# Nanopb: !! p' $$file; \
echo -n "]("; echo $$file | sed 's/.md/.html)/' ; done;) | \
pandoc -f markdown -t html5 >> $@
echo '</div>' >> $@
%.html: %.md tmp_menu.html
sed '1 s!#!%!' $< | \
pandoc -s -f markdown -t html5 -c lsr.css --toc --toc-depth=4 \
--variable 'header-includes=<link href="favicon.ico" type="image/x-icon" rel="shortcut icon" />' \
--indented-code-classes=c \
-o $@
sed -i '/<nav/e cat feedback.html' $@
sed -i 's/doc_page_name_placeholder/$</' $@
sed -i 's!<nav[^>]*>!\0<b>Contents:</b>!' $@
sed -i '/<nav/e cat tmp_menu.html' $@

61
lib/nanopb/docs/bazel_build.md Normal file
View File

@@ -0,0 +1,61 @@
# Nanopb: Bazel build
The Bazel build system, is designed to be fast and correct. Nanopb provides a
set of plugins for the Bazel build system allowing Nanopb to be integrated
into the build.
## Getting started
Add the following to your WORKSPACE file.
``` py
# WORKSPACE
git_repository(
name = "com_github_nanopb_nanopb",
remote = "https://github.com/nanopb/nanopb.git"
commit = "<TODO:Enter your desired commit>",
)
load("@com_github_nanopb_nanopb//extra/bazel:nanopb_deps.bzl", "nanopb_deps")
nanopb_deps()
load("@com_github_nanopb_nanopb//extra/bazel:python_deps.bzl",
"nanopb_python_deps")
nanopb_python_deps()
load("@com_github_nanopb_nanopb//extra/bazel:nanopb_workspace.bzl",
"nanopb_workspace")
nanopb_workspace()
```
To use the Nanopb rules with in your build you can use the
`cc_nanopb_proto_library` which works in a similar way to the native
`cc_proto_library` rule.
``` py
# BUILD.bazel
load("@com_github_nanopb_nanopb//extra/bazel:nanopb_cc_proto_library.bzl",
"nanopb_cc_proto_library")
# Your native proto_library.
proto_library(
name = "descriptor",
srcs = [
"generator/proto/google/protobuf/descriptor.proto",
],
)
# Generated library.
cc_nanopb_proto_library(
name = "descriptor_nanopb",
protos = [":descriptor"],
visibility = ["//visibility:private"],
)
# Depend directly on the generated code using a cc_library.
cc_library(
name = "uses_generated_descriptors",
deps = [":descriptor_nanopb"],
hdrs = ["my_header.h"],
)
```

622
lib/nanopb/docs/concepts.md Normal file
View File

@@ -0,0 +1,622 @@
# Nanopb: Basic concepts
The things outlined here are the underlying concepts of the nanopb
design.
## Proto files
All Protocol Buffers implementations use .proto files to describe the
message format. The point of these files is to be a portable interface
description language.
### Compiling .proto files for nanopb
Nanopb comes with a Python script to generate `.pb.c` and
`.pb.h` files from the `.proto` definition:
user@host:~$ nanopb/generator/nanopb_generator.py message.proto
Writing to message.pb.h and message.pb.c
Internally this script uses Google `protoc` to parse the
input file. If you do not have it available, you may receive an error
message. You can install either `grpcio-tools` Python
package using `pip`, or the `protoc` compiler
itself from `protobuf-compiler` distribution package.
Generally the Python package is recommended, because nanopb requires
protoc version 3.6 or newer to support all features, and some distributions come with an older
version.
### Modifying generator behaviour
Using generator options, you can set maximum sizes for fields in order
to allocate them statically. The preferred way to do this is to create
an .options file with the same name as your .proto file:
# Foo.proto
message Foo {
required string name = 1;
}
# Foo.options
Foo.name max_size:16
For more information on this, see the [Proto file
options](reference.html#proto-file-options) section in the reference
manual.
## Streams
Nanopb uses streams for accessing the data in encoded format. The stream
abstraction is very lightweight, and consists of a structure
(`pb_ostream_t` or `pb_istream_t`) which contains a pointer to a
callback function.
There are a few generic rules for callback functions:
1) Return false on IO errors. The encoding or decoding process will
abort immediately.
2) Use state to store your own data, such as a file descriptor.
3) `bytes_written` and `bytes_left` are updated by pb_write and
pb_read.
4) Your callback may be used with substreams. In this case
`bytes_left`, `bytes_written` and `max_size` have smaller values
than the original stream. Don't use these values to calculate
pointers.
5) Always read or write the full requested length of data. For example,
POSIX `recv()` needs the `MSG_WAITALL` parameter to accomplish
this.
### Output streams
struct _pb_ostream_t
{
bool (*callback)(pb_ostream_t *stream, const uint8_t *buf, size_t count);
void *state;
size_t max_size;
size_t bytes_written;
};
The `callback` for output stream may be NULL, in which case the stream
simply counts the number of bytes written. In this case, `max_size` is
ignored.
Otherwise, if `bytes_written` + bytes_to_be_written is larger than
`max_size`, pb_write returns false before doing anything else. If you
don\'t want to limit the size of the stream, pass SIZE_MAX.
**Example 1:**
This is the way to get the size of the message without storing it
anywhere:
Person myperson = ...;
pb_ostream_t sizestream = {0};
pb_encode(&sizestream, Person_fields, &myperson);
printf("Encoded size is %d\n", sizestream.bytes_written);
**Example 2:**
Writing to stdout:
bool callback(pb_ostream_t `stream, const uint8_t `buf, size_t count)
{
FILE *file = (FILE*) stream->state;
return fwrite(buf, 1, count, file) == count;
}
pb_ostream_t stdoutstream = {&callback, stdout, SIZE_MAX, 0};
### Input streams
For input streams, there is one extra rule:
6) You don't need to know the length of the message in advance. After
getting EOF error when reading, set `bytes_left` to 0 and return
`false`. `pb_decode()` will detect this and if the EOF was in a proper
position, it will return true.
Here is the structure:
struct _pb_istream_t
{
bool (*callback)(pb_istream_t *stream, uint8_t *buf, size_t count);
void *state;
size_t bytes_left;
};
The `callback` must always be a function pointer. `Bytes_left` is an
upper limit on the number of bytes that will be read. You can use
SIZE_MAX if your callback handles EOF as described above.
**Example:**
This function binds an input stream to stdin:
bool callback(pb_istream_t *stream, uint8_t *buf, size_t count)
{
FILE *file = (FILE*)stream->state;
bool status;
if (buf == NULL)
{
while (count-- && fgetc(file) != EOF);
return count == 0;
}
status = (fread(buf, 1, count, file) == count);
if (feof(file))
stream->bytes_left = 0;
return status;
}
pb_istream_t stdinstream = {&callback, stdin, SIZE_MAX};
## Data types
Most Protocol Buffers datatypes have directly corresponding C datatypes,
such as `int32` is `int32_t`, `float` is `float` and `bool` is `bool`. However, the
variable-length datatypes are more complex:
1) Strings, bytes and repeated fields of any type map to callback
functions by default.
2) If there is a special option `(nanopb).max_size` specified in the
.proto file, string maps to null-terminated char array and bytes map
to a structure containing a char array and a size field.
3) If `(nanopb).fixed_length` is set to `true` and
`(nanopb).max_size` is also set, then bytes map to an inline byte
array of fixed size.
4) If there is a special option `(nanopb).max_count` specified on a
repeated field, it maps to an array of whatever type is being
repeated. Another field will be created for the actual number of
entries stored.
5) If `(nanopb).fixed_count` is set to `true` and
`(nanopb).max_count` is also set, the field for the actual number
of entries will not by created as the count is always assumed to be
max count.
### Examples of .proto specifications vs. generated structure
**Simple integer field:**\
.proto: `int32 age = 1;`\
.pb.h: `int32_t age;`
**String with unknown length:**\
.proto: `string name = 1;`\
.pb.h: `pb_callback_t name;`
**String with known maximum length:**\
.proto: `string name = 1 [(nanopb).max_length = 40];`\
.pb.h: `char name[41];`
**Repeated string with unknown count:**\
.proto: `repeated string names = 1;`\
.pb.h: `pb_callback_t names;`
**Repeated string with known maximum count and size:**\
.proto: `repeated string names = 1 [(nanopb).max_length = 40, (nanopb).max_count = 5];`\
.pb.h: `size_t names_count;` `char names[5][41];`
**Bytes field with known maximum size:**\
.proto: `bytes data = 1 [(nanopb).max_size = 16];`\
.pb.h: `PB_BYTES_ARRAY_T(16) data;`, where the struct contains `{pb_size_t size; pb_byte_t bytes[n];}`
**Bytes field with fixed length:**\
.proto: `bytes data = 1 [(nanopb).max_size = 16, (nanopb).fixed_length = true];`\
.pb.h: `pb_byte_t data[16];`
**Repeated integer array with known maximum size:**\
.proto: `repeated int32 numbers = 1 [(nanopb).max_count = 5];`\
.pb.h: `pb_size_t numbers_count;` `int32_t numbers[5];`
**Repeated integer array with fixed count:**\
.proto: `repeated int32 numbers = 1 [(nanopb).max_count = 5, (nanopb).fixed_count = true];`\
.pb.h: `int32_t numbers[5];`
The maximum lengths are checked in runtime. If string/bytes/array
exceeds the allocated length, `pb_decode()` will return false.
> **Note:** For the `bytes` datatype, the field length checking may not be
exact. The compiler may add some padding to the `pb_bytes_t`
structure, and the nanopb runtime doesn't know how much of the
structure size is padding. Therefore it uses the whole length of the
structure for storing data, which is not very smart but shouldn't cause
problems. In practise, this means that if you specify
`(nanopb).max_size=5` on a `bytes` field, you may be able to store 6
bytes there. For the `string` field type, the length limit is exact.
> **Note:** The decoder only keeps track of one `fixed_count` repeated field at a time. Usually this it not an issue because all elements of a repeated field occur end-to-end. Interleaved array elements of several `fixed_count` repeated fields would be a valid protobuf message, but would get rejected by nanopb decoder with error `"wrong size for fixed count field"`.
## Field callbacks
The easiest way to handle repeated fields is to specify a maximum size for
them, as shown in the previous section. However, sometimes you need to be
able to handle arrays with unlimited length, possibly larger than available
RAM memory.
For these cases, nanopb provides a callback interface. Nanopb core invokes
the callback function when it gets to the specific field in the message.
Your code can then handle the field in custom ways, for example decode
the data piece-by-piece and store to filesystem.
The [pb_callback_t](reference.html#pb-callback-t) structure contains a
function pointer and a `void` pointer called `arg` you can use for
passing data to the callback. If the function pointer is NULL, the field
will be skipped. A pointer to the `arg` is passed to the function, so
that it can modify it and retrieve the value.
The actual behavior of the callback function is different in encoding
and decoding modes. In encoding mode, the callback is called once and
should write out everything, including field tags. In decoding mode, the
callback is called repeatedly for every data item.
To write more complex field callbacks, it is recommended to read the
[Google Protobuf Encoding Specification](https://developers.google.com/protocol-buffers/docs/encoding).
### Encoding callbacks
bool (*encode)(pb_ostream_t *stream, const pb_field_iter_t *field, void * const *arg);
| | |
| ---------- | ------------------------------------------------------------------ |
| `stream` | Output stream to write to |
| `field` | Iterator for the field currently being encoded or decoded. |
| `arg` | Pointer to the `arg` field in the `pb_callback_t` structure. |
When encoding, the callback should write out complete fields, including
the wire type and field number tag. It can write as many or as few
fields as it likes. For example, if you want to write out an array as
`repeated` field, you should do it all in a single call.
Usually you can use [pb_encode_tag_for_field](reference.html#pb-encode-tag-for-field) to
encode the wire type and tag number of the field. However, if you want
to encode a repeated field as a packed array, you must call
[pb_encode_tag](reference.html#pb-encode-tag) instead to specify a
wire type of `PB_WT_STRING`.
If the callback is used in a submessage, it will be called multiple
times during a single call to [pb_encode](reference.html#pb-encode). In
this case, it must produce the same amount of data every time. If the
callback is directly in the main message, it is called only once.
This callback writes out a dynamically sized string:
bool write_string(pb_ostream_t *stream, const pb_field_iter_t *field, void * const *arg)
{
char *str = get_string_from_somewhere();
if (!pb_encode_tag_for_field(stream, field))
return false;
return pb_encode_string(stream, (uint8_t*)str, strlen(str));
}
### Decoding callbacks
bool (*decode)(pb_istream_t *stream, const pb_field_iter_t *field, void **arg);
| | |
| ---------- | ------------------------------------------------------------------ |
| `stream` | Input stream to read from |
| `field` | Iterator for the field currently being encoded or decoded. |
| `arg` | Pointer to the `arg` field in the `pb_callback_t` structure. |
When decoding, the callback receives a length-limited substring that
reads the contents of a single field. The field tag has already been
read. For `string` and `bytes`, the length value has already been
parsed, and is available at `stream->bytes_left`.
The callback will be called multiple times for repeated fields. For
packed fields, you can either read multiple values until the stream
ends, or leave it to [pb_decode](reference.html#pb-decode) to call your
function over and over until all values have been read.
This callback reads multiple integers and prints them:
bool read_ints(pb_istream_t *stream, const pb_field_iter_t *field, void **arg)
{
while (stream->bytes_left)
{
uint64_t value;
if (!pb_decode_varint(stream, &value))
return false;
printf("%lld\n", value);
}
return true;
}
### Function name bound callbacks
bool MyMessage_callback(pb_istream_t *istream, pb_ostream_t *ostream, const pb_field_iter_t *field);
| | |
| ---------- | ------------------------------------------------------------------ |
| `istream` | Input stream to read from, or NULL if called in encoding context. |
| `ostream` | Output stream to write to, or NULL if called in decoding context. |
| `field` | Iterator for the field currently being encoded or decoded. |
Storing function pointer in `pb_callback_t` fields inside
the message requires extra storage space and is often cumbersome. As an
alternative, the generator options `callback_function` and
`callback_datatype` can be used to bind a callback function
based on its name.
Typically this feature is used by setting `callback_datatype` to e.g. `void\*` or even a struct type used to store encoded or decoded data.
The generator will automatically set `callback_function` to `MessageName_callback` and produce a prototype for it in generated `.pb.h`.
By implementing this function in your own code, you will receive callbacks for fields without having to separately set function pointers.
If you want to use function name bound callbacks for some fields and
`pb_callback_t` for other fields, you can call
`pb_default_field_callback` from the message-level
callback. It will then read a function pointer from
`pb_callback_t` and call it.
## Message descriptor
For using the `pb_encode()` and `pb_decode()` functions, you need a
description of all the fields contained in a message. This description
is usually autogenerated from .proto file.
For example this submessage in the Person.proto file:
~~~~ protobuf
message Person {
message PhoneNumber {
required string number = 1 [(nanopb).max_size = 40];
optional PhoneType type = 2 [default = HOME];
}
}
~~~~
This in turn generates a macro list in the `.pb.h` file:
#define Person_PhoneNumber_FIELDLIST(X, a) \
X(a, STATIC, REQUIRED, STRING, number, 1) \
X(a, STATIC, OPTIONAL, UENUM, type, 2)
Inside the `.pb.c` file there is a macro call to
`PB_BIND`:
PB_BIND(Person_PhoneNumber, Person_PhoneNumber, AUTO)
These macros will in combination generate `pb_msgdesc_t`
structure and associated lists:
const uint32_t Person_PhoneNumber_field_info[] = { ... };
const pb_msgdesc_t * const Person_PhoneNumber_submsg_info[] = { ... };
const pb_msgdesc_t Person_PhoneNumber_msg = {
2,
Person_PhoneNumber_field_info,
Person_PhoneNumber_submsg_info,
Person_PhoneNumber_DEFAULT,
NULL,
};
The encoding and decoding functions take a pointer to this structure and
use it to process each field in the message.
## Oneof
Protocol Buffers supports
[oneof](https://developers.google.com/protocol-buffers/docs/reference/proto2-spec#oneof_and_oneof_field)
sections, where only one of the fields contained within can be present. Here is an example of `oneof` usage:
~~~~ protobuf
message MsgType1 {
required int32 value = 1;
}
message MsgType2 {
required bool value = 1;
}
message MsgType3 {
required int32 value1 = 1;
required int32 value2 = 2;
}
message MyMessage {
required uint32 uid = 1;
required uint32 pid = 2;
required uint32 utime = 3;
oneof payload {
MsgType1 msg1 = 4;
MsgType2 msg2 = 5;
MsgType3 msg3 = 6;
}
}
~~~~
Nanopb will generate `payload` as a C union and add an additional field
`which_payload`:
typedef struct _MyMessage {
uint32_t uid;
uint32_t pid;
uint32_t utime;
pb_size_t which_payload;
union {
MsgType1 msg1;
MsgType2 msg2;
MsgType3 msg3;
} payload;
} MyMessage;
`which_payload` indicates which of the `oneof` fields is actually set.
The user is expected to set the field manually using the correct field
tag:
MyMessage msg = MyMessage_init_zero;
msg.payload.msg2.value = true;
msg.which_payload = MyMessage_msg2_tag;
Notice that neither `which_payload` field nor the unused fields in
`payload` will consume any space in the resulting encoded message.
When a field inside `oneof` contains `pb_callback_t`
fields, the callback values cannot be set before decoding. This is
because the different fields share the same storage space in C
`union`. Instead either function name bound callbacks or a
separate message level callback can be used. See
[tests/oneof_callback](https://github.com/nanopb/nanopb/tree/master/tests/oneof_callback)
for an example on this.
## Extension fields
Protocol Buffers supports a concept of [extension
fields](https://developers.google.com/protocol-buffers/docs/proto#extensions),
which are additional fields to a message, but defined outside the actual
message. The definition can even be in a completely separate .proto
file.
The base message is declared as extensible by keyword `extensions` in
the .proto file:
~~~~ protobuf
message MyMessage {
.. fields ..
extensions 100 to 199;
}
~~~~
For each extensible message, `nanopb_generator.py` declares an
additional callback field called `extensions`. The field and associated
datatype `pb_extension_t` forms a linked list of handlers. When an
unknown field is encountered, the decoder calls each handler in turn
until either one of them handles the field, or the list is exhausted.
The actual extensions are declared using the `extend` keyword in the
.proto, and are in the global namespace:
~~~~ protobuf
extend MyMessage {
optional int32 myextension = 100;
}
~~~~
For each extension, `nanopb_generator.py` creates a constant of type
`pb_extension_type_t`. To link together the base message and the
extension, you have to:
1. Allocate storage for your field, matching the datatype in the
.proto. For example, for a `int32` field, you need a `int32_t`
variable to store the value.
2. Create a `pb_extension_t` constant, with pointers to your variable
and to the generated `pb_extension_type_t`.
3. Set the `message.extensions` pointer to point to the
`pb_extension_t`.
An example of this is available in `tests/test_encode_extensions.c`
and `tests/test_decode_extensions.c`.
## Default values
Protobuf has two syntax variants, proto2 and proto3. Of these proto2 has
user definable default values that can be given in .proto file:
~~~~ protobuf
message MyMessage {
optional bytes foo = 1 [default = "ABC\x01\x02\x03"];
optional string bar = 2 [default = "åäö"];
}
~~~~
Nanopb will generate both static and runtime initialization for the
default values. In `myproto.pb.h` there will be a
`#define MyMessage_init_default {...}` that can be used to initialize
whole message into default values:
MyMessage msg = MyMessage_init_default;
In addition to this, `pb_decode()` will initialize message
fields to defaults at runtime. If this is not desired,
`pb_decode_ex()` can be used instead.
## Message framing
Protocol Buffers does not specify a method of framing the messages for
transmission. This is something that must be provided by the library
user, as there is no one-size-fits-all solution. Typical needs for a
framing format are to:
1. Encode the message length.
2. Encode the message type.
3. Perform any synchronization and error checking that may be needed
depending on application.
For example UDP packets already fulfill all the requirements, and TCP
streams typically only need a way to identify the message length and
type. Lower level interfaces such as serial ports may need a more robust
frame format, such as HDLC (high-level data link control).
Nanopb provides a few helpers to facilitate implementing framing
formats:
1. Functions `pb_encode_ex` and `pb_decode_ex` prefix the message
data with a varint-encoded length.
2. Union messages and oneofs are supported in order to implement
top-level container messages.
3. Message IDs can be specified using the `(nanopb_msgopt).msgid`
option and can then be accessed from the header.
## Return values and error handling
Most functions in nanopb return bool: `true` means success, `false`
means failure. There is also support for error messages for
debugging purposes: the error messages go in `stream->errmsg`.
The error messages help in guessing what is the underlying cause of the
error. The most common error conditions are:
1) Invalid protocol buffers binary message.
2) Mismatch between binary message and .proto message type.
3) Unterminated message (incorrect message length).
4) Exceeding the max_size or bytes_left of a stream.
5) Exceeding the max_size/max_count of a string or array field
6) IO errors in your own stream callbacks.
7) Errors that happen in your callback functions.
8) Running out of memory, i.e. stack overflow.
9) Invalid field descriptors (would usually mean a bug in the generator).
## Static assertions
Nanopb code uses static assertions to check size of structures at the compile
time. The `PB_STATIC_ASSERT` macro is defined in `pb.h`. If ISO C11 standard
is available, the C standard `_Static_assert` keyword is used, otherwise a
negative sized array definition trick is used.
Common reasons for static assertion errors are:
1. `FIELDINFO_DOES_NOT_FIT_width2` with `width1` or `width2`:
Message that is larger than 256 bytes, but nanopb generator does not detect
it for some reason. Often resolved by giving all `.proto` files as argument
to `nanopb_generator.py` at the same time, to ensure submessage definitions
are found. Alternatively `(nanopb).descriptorsize = DS_4` option can be
given manually.
2. `FIELDINFO_DOES_NOT_FIT_width4` with `width4`:
Message that is larger than 64 kilobytes. There will be a better error
message for this in a future nanopb version, but currently it asserts here.
The compile time option `PB_FIELD_32BIT` should be specified either on
C compiler command line or by editing `pb.h`. This will increase the sizes
of integer types used internally in nanopb code.
3. `DOUBLE_MUST_BE_8_BYTES`:
Some platforms, most notably AVR, do not support the 64-bit `double` type,
only 32-bit `float`. The compile time option `PB_CONVERT_DOUBLE_FLOAT` can
be defined to convert between the types automatically. The conversion
results in small rounding errors and takes unnecessary space in transmission,
so changing the `.proto` to use `float` type is often better.
4. `INT64_T_WRONG_SIZE`:
The `stdint.h` system header is incorrect for the C compiler being used.
This can result from erroneous compiler include path.
If the compiler actually does not support 64-bit types, the compile time
option `PB_WITHOUT_64BIT` can be used.
5. `variably modified array size`:
The compiler used has problems resolving the array-based static assert at
compile time. Try setting the compiler to C11 standard mode if possible.
If static assertions cannot be made to work on the compiler used, the
compile-time option `PB_NO_STATIC_ASSERT` can be specified to turn them off.

2869
lib/nanopb/docs/generator_flow.svg Normal file
View File

File diff suppressed because it is too large Load Diff
Side by Side

After

Width:  |  Height:  |  Size: 112 KiB

158
lib/nanopb/docs/index.md Normal file
View File

@@ -0,0 +1,158 @@
# Nanopb: Overview
Nanopb is an ANSI-C library for encoding and decoding messages in
Google's [Protocol Buffers](https://developers.google.com/protocol-buffers/docs/reference/overview)
format with minimal requirements for RAM and code space. It is primarily
suitable for 32-bit microcontrollers.
Documentation version
---------------------
This documentation applies for nanopb 0.4.0 and later versions. For
documentation of older releases,
[see here](https://github.com/nanopb/nanopb/blob/maintenance_0.3/docs/index.rst).
Overall structure
-----------------
For the runtime program, you always need `pb.h` for type declarations
and `pb_common.h/c` for base functions. Depending on whether you want
to encode, decode, or both, you also need `pb_encode.h/c` or
`pb_decode.h/c`.
The high-level encoding and decoding functions take a pointer to
`pb_msgdesc_t` structure, which describes the fields of a message
structure. Usually you want these autogenerated from a `.proto` file.
The tool script `nanopb_generator.py` accomplishes this.
![Image: Nanopb generator flow](generator_flow.svg)
So a typical project might include these files:
1. Nanopb runtime library:
- pb.h
- pb_common.h and pb_common.c (always needed)
- pb_decode.h and pb_decode.c (needed for decoding messages)
- pb_encode.h and pb_encode.c (needed for encoding messages)
2. Protocol description (you can have many):
- person.proto (just an example)
- person.pb.c (autogenerated, contains message descriptors)
- person.pb.h (autogenerated, contains type declarations and macros)
Features and limitations
------------------------
**Features**
1) Pure C runtime
2) Small code size (5--20 kB depending on processor and compilation options, plus any message definitions)
3) Small ram usage (typically \~1 kB stack, plus any message structs)
4) Allows specifying maximum size for strings and arrays, so that they can be allocated statically.
5) No malloc needed: everything can be allocated statically or on the stack. Optional malloc support available.
6) You can use either encoder or decoder alone to cut the code size in half.
7) Support for most protobuf features, including: all data types,
nested submessages, default values, repeated and optional fields,
oneofs, packed arrays, extension fields.
8) Callback mechanism for handling messages larger than can fit in available RAM.
9) Extensive set of tests.
**Limitations**
1) Some speed has been sacrificed for code size.
2) Encoding is focused on writing to streams. For memory buffers only it could be made more efficient.
3) The deprecated Protocol Buffers feature called "groups" is not supported.
4) Fields in the generated structs are ordered by the tag number, instead of the natural ordering in .proto file. (Since nanopb-0.4.2 this can be configured with `sort_by_tag` setting.)
5) Unknown fields are not preserved when decoding and re-encoding a message.
6) Reflection (runtime introspection) is not supported. E.g. you can't request a field by giving its name in a string.
7) Numeric arrays are always encoded as packed, even if not marked as packed in .proto.
8) Cyclic references between messages are supported only in callback and malloc mode.
9) Nanopb doesn't have a stable ABI (application binary interface)
between versions, so using it as a shared library (.so / .dll)
requires extra care.
Getting started
---------------
For starters, consider this simple message:
~~~~ protobuf
message Example {
required int32 value = 1;
}
~~~~
Save this in `message.proto` and compile it:
user@host:~$ python nanopb/generator/nanopb_generator.py message.proto
You should now have in `message.pb.h`:
typedef struct {
int32_t value;
} Example;
extern const pb_msgdesc_t Example_msg;
#define Example_fields &Example_msg
Then you have to include the nanopb headers and the generated header:
#include <pb_encode.h>
#include "message.pb.h"
Now in your main program do this to encode a message:
Example mymessage = {42};
uint8_t buffer[10];
pb_ostream_t stream = pb_ostream_from_buffer(buffer, sizeof(buffer));
pb_encode(&stream, Example_fields, &mymessage);
After that, buffer will contain the encoded message. The number of bytes
in the message is stored in `stream.bytes_written`. You can feed the
message to `protoc --decode=Example message.proto` to verify its
validity.
For a complete example of the simple case, see [examples/simple/simple.c](https://github.com/nanopb/nanopb/blob/master/examples/simple/simple.c).
For a more complex example with network interface, see the [examples/network_server](https://github.com/nanopb/nanopb/tree/master/examples/network_server) subdirectory.
Compiler requirements
---------------------
Nanopb should compile with most ansi-C compatible compilers. It however
requires a few header files to be available:
1) `string.h`, with these functions: `strlen`, `memcpy`, `memset`
2) `stdint.h`, for definitions of `int32_t` etc.
3) `stddef.h`, for definition of `size_t`
4) `stdbool.h`, for definition of `bool`
5) `limits.h`, for definition of `CHAR_BIT`
If these header files do not come with your compiler, you can use the
file `extra/pb_syshdr.h` instead. It contains an example of how to
provide the dependencies. You may have to edit it a bit to suit your
custom platform.
To use the pb_syshdr.h, define `PB_SYSTEM_HEADER` as
`"pb_syshdr.h"` (including the quotes). Similarly, you can provide a
custom include file, which should provide all the dependencies listed
above.
Running the test cases
----------------------
Extensive unittests and test cases are included under the `tests`
folder.
To build the tests, you will need the [scons](http://www.scons.org/)
build system. The tests should be runnable on most platforms. Windows
and Linux builds are regularly tested. The tests also support embedded
targets: STM32 (ARM Cortex-M) and AVR builds are regularly tested.
In addition to the build system, you will also need a working Google
Protocol Buffers `protoc` compiler, and the Python bindings for Protocol
Buffers.
Easiest way to install dependencies is to use the Python package manager
[pip](https://pypi.org/project/pip/), which works on all platforms supported by Python:
pip3 install scons protobuf grpcio-tools

BIN
lib/nanopb/docs/logo/logo.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 15 KiB

1470
lib/nanopb/docs/logo/logo.svg Normal file
View File

File diff suppressed because one or more lines are too long
Side by Side

After

Width:  |  Height:  |  Size: 102 KiB

BIN
lib/nanopb/docs/logo/logo16px.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 854 B

BIN
lib/nanopb/docs/logo/logo48px.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.5 KiB

233
lib/nanopb/docs/lsr.css Normal file
View File

@@ -0,0 +1,233 @@
/*
Original author: Peter Parente
Date: 2008/01/22
Version: 1.0 (modified)
Copyright: This stylesheet has been placed in the public domain - free to edit and use for all uses.
--
Heavily modified for use in nanopb documentation.
2011-2020 Petteri Aimonen
*/
body {
font: 100% sans-serif;
background: #ffffff;
color: black;
margin: 2em;
padding: 0em 2em;
max-width: 1200px;
}
p.topic-title {
font-weight: bold;
}
table.docinfo {
text-align: left;
margin: 2em 0em;
}
a[href] {
color: #436976;
background-color: transparent;
}
a.toc-backref {
text-decoration: none;
}
h1 a[href] {
color: #003a6b;
text-decoration: none;
background-color: transparent;
}
a.strong {
font-weight: bold;
}
img {
margin: 0;
border: 0;
}
blockquote {
border-left: 3px solid black;
background-color: #f6f6f6;
padding: 0.5em 1em 0.2em 1em;
}
li {
margin-bottom: 0.5em;
}
p {
margin: 0.5em 0 1em 0;
line-height: 1.5em;
}
p a:visited {
color: purple;
background-color: transparent;
}
p a:active {
color: red;
background-color: transparent;
}
a:hover {
text-decoration: none;
}
p img {
border: 0;
margin: 0;
}
p.rubric {
font-weight: bold;
font-style: italic;
}
em, cite {
font-style: normal;
font-family: monospace;
font-weight: bold;
}
pre {
border-left: 3px double #aaa;
padding: 5px 10px;
background-color: #f6f6f6;
}
code {
border: 1px solid rgba(128, 128, 128, 0.1);
padding: 1px 5px;
border-radius: 5px;
background-color: rgba(128, 128, 128, 0.05);
white-space: nowrap;
}
pre code {
border: none;
background-color: transparent;
padding: 0px;
}
h1, h2, h3, h4, h5, h6 {
color: #000;
background-color: transparent;
margin: 0em;
padding-top: 0.5em;
}
h1 {
color: #003a6b;
font-size: 180%;
margin-bottom: 0.5em;
border-bottom: 2px solid #aaa;
}
h2 {
color: #003a6b;
font-size: 130%;
margin-bottom: 0.5em;
border-bottom: 1px solid #aaa;
margin-top: 1.5em;
}
h3 {
font-size: 120%;
margin-bottom: 0.5em;
margin-top: 1.0em;
}
h4 {
font-size: 110%;
font-weight: bold;
margin-bottom: 0.5em;
margin-top: 0.5em;
}
h5 {
font-size: 105%;
font-weight: bold;
margin-bottom: 0.5em;
}
h6 {
font-size: 100%;
font-weight: bold;
margin-bottom: 0.5em;
}
dt {
font-style: italic;
}
dd {
margin-bottom: 1.5em;
}
table {
text-align: left;
border-collapse: collapse;
margin: 1.5em 0em;
}
table td, table th {
padding: 0.25em 1em;
border-top: 1px solid gray;
border-bottom: 1px solid gray;
}
#index {
margin: 2em 2em 2em 0em;
padding: 0em 1em;
border-top: 1px solid #aaa;
border-left: 1px solid #aaa;
border-bottom: 2px solid #555;
border-right: 2px solid #555;
max-width: 20em;
}
#index h2 {
margin-bottom: 0em;
margin-top: 0em;
color: #003a6b;
font-size: 100%;
border-bottom: 1px solid #aaa;
font-weight: bold;
}
#feedback_link {
float: right;
}
#feedback {
visibility: hidden;
padding: 1em;
border-radius: 5px;
position: fixed;
top: 10em;
right: 10em;
background: white;
border-top: 1px solid #aaa;
border-left: 1px solid #aaa;
border-bottom: 2px solid #555;
border-right: 2px solid #555;
}
#feedback:target {
visibility: visible;
}
#feedback .cancel {
color: #666;
padding-left: 2em;
}

663
lib/nanopb/docs/migration.md Normal file
View File

@@ -0,0 +1,663 @@
# Nanopb: Migration from older versions
This document details all the breaking changes that have been made to
nanopb since its initial release. For each change, the rationale and
required modifications of user applications are explained. Also any
error indications are included, in order to make it easier to find this
document.
Nanopb-0.4.7 (2022-xx-xx)
-------------------------
### Updated include path order in FindNanopb.cmake
**Changes:** The include path passed to `protoc` by the CMake rules was updated.
**Required actions:** No changes needed for most users.
In some specific cases it could change the directory hierarchy generated by `protoc`.
More details in
[pull request #822](https://github.com/nanopb/nanopb/pull/822).
**Error indications:** Generated `.pb.c` or `.pb.h` file not found when building
with CMake rules.
Nanopb-0.4.6 (2022-05-30)
-------------------------
### NANOPB_VERSION define is now a string
**Changes:** To ease `NANOPB_VERSION` macro usage, the value is directly a string.
**Required actions:** Most nanopb users probably never used that macro. If so,
you certainly use the `#` preprocessor to convert it as string. You, now,
only have to call it directly, like this for example:
`strcpy(myvar, NANOPB_VERSION);`
### FindNanopb.cmake now requires protoc 3.6.0 or newer by default
**Changes:** The default options passing method now uses `--plugin-opt` which
is supported by protoc 3.6.0 and newer (released in 2018).
**Required actions:** Update `protoc` if needed, or alternatively install
`grpcio-tools` package from `pip`. If neither is possible, the
`NANOPB_PROTOC_OLDER_THAN_3_6_0` cmake option can be used to restore the old
style option passing. Note that it has problems with special characters such
as `:`.
**Error indications:** "`protoc: Unknown flag: --nanopb_opt`"
### pb.h uses C11 _Static_assert keyword by default
**Rationale:** The nanopb generated headers use static assertions to catch
errors at compile time. There are several mechanisms to implement this.
The most widely supported is C11 `_Static_assert` keyword.
Previously the code used negative size array definition trick, which is
supported already in C99 but does not work with every compiler and can
produce confusing error messages.
**Changes:** Now `_Static_assert` is used by default.
**Required actions:** If the keyword is not recognized, set the compiler to
C11 standard mode if available. If it is not available, define either `PB_C99_STATIC_ASSERT`
or `PB_NO_STATIC_ASSERT` in `pb.h` or on compiler command line.
**Error indications:** `Undefined identifier _Static_assert`
Nanopb-0.4.4 (2020-11-25)
-------------------------
### Remove outdated generator/nanopb/options.proto
**Changes:** Back in 2018, it was considered in pull request #241 to
move nanopb generator options to a separate namespace. For this reason,
a transitional file was added. It was later abandoned and is now removed
to avoid confusion.
**Required actions:** Most nanopb users probably never used that transitional
file at all. If your `.proto` files import it, change to using `generator/proto/nanopb.proto`.
**Error indications:** Errors about missing file `options.proto` when running
the generator.
Nanopb-0.4.3 (2020-09-21)
-------------------------
### pb_msgdesc_t struct has new fields
**Changes:** New fields `required_field_count` and
`largest_tag` were added to `pb_msgdesc_t`
and existing fields were reordered.
**Required actions:** All `.pb.c` files must be recompiled.
Regeneration is not needed.
**Error indications:** Messages may fail to encode or decode, or the
code can crash inside `load_descriptor_values()` in
`pb_common.c`.
Nanopb-0.4.2 (2020-06-23)
-------------------------
### Generator now uses Python 3 by default
**Rationale:** Previously `nanopb-generator.py` had hashbang
of `#!/usr/bin/env python`, which would execute with Python
2 on most systems. Python 2 is now deprecated and many libraries are
dropping support for it, which makes installing dependencies difficult.
While `nanopb_generator.py` has worked with Python 3 for
years now, and overriding the python version was possible with
virtualenv, that was an extra complication.
**Changes:** Hashbang now uses `#!/usr/bin/env python3`.
New file `nanopb_generator.py2` can be used to run with
Python 2, if necessary.
**Required actions:** If possible, just verify Python 3 is installed and
necessary dependencies are installed for it. For example `pip3 install protobuf grpcio-tools`
should take care of it. If this is not possible, call `nanopb_generator.py2` from your build
scripts instead.
**Error indications:** `python3: command not found` if
Python 3 is not installed.
`Could not import the Google protobuf Python libraries` if dependencies are only installed for Python 2.
Nanopb-0.4.0 (2019-12-20)
-------------------------
### New field descriptor format
**Rationale:** Previously information about struct fields was stored as
an array of `pb_field_t` structures. This was a
straightforward method, but required allocating space for e.g.
submessage type and array size for all fields, even though most fields
are not submessages nor arrays.
**Changes:** Now field information is encoded more efficiently in
`uint32_t` array in a variable-length format. Old
`pb_field_t` structure has been removed and it is now a
typedef for `pb_field_iter_t`. This retains compatibility
with most old callback definitions. The field definitions in
`.pb.h` files are now of type `pb_msgdesc_t`.
**Required actions:** If your own code accesses the low-level field
information in `pb_field_t`, it must be modified to do so
only through the functions declared in `pb_common.h`.
**Error indications:** `incompatible pointer type` errors
relating to `pb_field_t`
### Changes to generator default options
**Rationale:** Previously nanopb_generator added a timestamp header to
generated files and used only basename of files in
`#include` directives. This is different than what the
`protoc` C++ backend does.
**Changes:** Now default options are `--no-timestamp` and
`--no-strip-path`.
**Required actions:** If old behaviour is desired, add
`--timestamp` and `--strip-path` options to
`nanopb_generator.py` or on `protoc` command
line as `--nanopb_out=--timestamp,--strip-path:outdir`.
**Error indications:** Compiler error: cannot find include file
`mymessage.pb.h` when compiling
`mymessage.pb.c`.
### Removal of bundled plugin.proto
**Rationale:** Google's Python protobuf library, which is used in
nanopb generator, has included `plugin_pb2` with it since
version 3.1.0. It is not necessary to bundle it with nanopb anymore.
**Required actions:** Update `python-protobuf` to version
3.1.0 or newer.
**Error indications:** `ImportError: No module named compiler.plugin_pb2`
### .options file is now always case-sensitive
**Rationale:** Previously field names in `.options` file
were case-sensitive on Linux and case-insensitive on Windows. This was
by accident. Because `.proto` files are case-sensitive,
`.options` files should be too.
**Changes:** Now field names in `.options` are always
case-sensitive, and matched by `fnmatchcase()` instead of
`fnmatch()`.
**Required actions:** If field names in `.options` are not
capitalized the same as in `.proto`, they must be updated.
### `CHAR_BIT` define is now needed
**Rationale:** To check whether the platform has 8-bit or larger chars,
the C standard `CHAR_BIT` macro is needed.
**Changes:** `pb.h` now includes `limits.h` for this macro.
**Required actions:** If your platform doesn't have `limits.h`
available, you can define the macro in `pb_syshdr.h`. There is an
example in `extra` directory.
**Error indications:** `"Cannot find include file <limits.h>."` or
`"Undefined identifier: CHAR_BIT."`
### Strings must now always be null-terminated
**Rationale:** Previously `pb_encode()` would accept non-terminated
strings and assume that they are the full length of the defined array.
However, `pb_decode()` would reject such messages because null
terminator wouldn't fit in the array.
**Changes:** `pb_encode()` will now return an error if null terminator
is missing. Maximum encoded message size calculation is changed
accordingly so that at most `max_size-1` strings are assumed. New field
option `max_length` can be used to define the maximum string length,
instead of the array size.
**Required actions:** If your strings were previously filling the whole
allocated array, increase the size of the field by 1.
**Error indications:** `pb_encode()` returns error `unterminated string`.
### Removal of per-field default value constants
**Rationale:** Previously nanopb declared a
`fieldname_default` constant variable for each field with a
default value, and used these internally to initialize messages. This
however used unnecessarily large amount of storage for the values. The
variables were mostly for internal usage, but were available in the
header file.
**Changes:** Default values are now stored as an encoded protobuf
message.
**Required actions:** If your code previously used default constants, it
will have to be adapted to take the default value in some other way,
such as by defining
`static const MyMessage msg_default = MyMessage_init_default;` and accessing
`msg_default.fieldname`.
**Error indications:** Compiler error about `fieldname_default` being undeclared.
### Zero tag in message now raises error by default
**Rationale:** Previously nanopb has allowed messages to be terminated
by a null byte, which is read as zero tag value. Most other protobuf
implementations don't support this, so it is not very useful feature.
It has also been noted that this can complicate debugging issues with
corrupted messages.
**Changes:** `pb_decode()` now gives error when it
encounters zero tag value. A new function `pb_decode_ex()`
supports flag `PB_DECODE_NULLTERMINATED` that supports
decoding null terminated messages.
**Required actions:** If application uses null termination for messages,
switch it to use `pb_decode_ex()` and
`pb_encode_ex()`. If compatibility with 0.3.9.x is needed,
there are also `pb_decode_nullterminated()` and
`pb_encode_nullterminated()` macros, which work both in
0.4.0 and 0.3.9.
**Error indications:** Error message from `pb_decode()`: `zero_tag`.
### Submessages now have has_field in proto3 mode
**Rationale:** Previously nanopb considered proto3 submessages as
present only when their contents was non-zero. Most other protobuf
libraries allow explicit null state for submessages.
**Changes:** Submessages now have separate `has_field` in
proto3 mode also.
**Required actions:** When using submessages in proto3 mode, user code
must now set `mymsg.has_submsg = true` for each submessage
that is present. Alternatively, the field option
`proto3_singular_msgs` can be used to restore the old
behavior.
**Error indications:** Submessages do not get encoded.
### PB_OLD_CALLBACK_STYLE option has been removed
**Rationale:** Back in 2013, function signature for callbacks was
changed. The `PB_OLD_CALLBACK_STYLE` option allowed
compatibility with old code, but complicated code and testing because of
the different options.
**Changes:** `PB_OLD_CALLBACK_STYLE` option no-longer has
any effect.
**Required actions:** If `PB_OLD_CALLBACK_STYLE` option
was in use previously, function signatures must be updated to use double
pointers (`void**` and `void * const *`).
**Error indications:** Assignment from incompatible pointer type.
### protoc insertion points are no longer included by default
**Rationale:** Protoc allows including comments in form
`@@protoc_insertion_point` to identify locations for
other plugins to insert their own extra content. Previously these were
included by default, but they clutter the generated files and are rarely
used.
**Changes:** Insertion points are now included only when
`--protoc-insertion-points` option is passed to the
generator.
Nanopb-0.3.9.4, 0.4.0 (2019-10-13)
----------------------------------
### Fix generation of min/max defines for enum types
**Rationale:** Nanopb generator makes \#defines for enum minimum and
maximum value. Previously these defines incorrectly had the first and
last enum value, instead of the actual minimum and maximum. (issue
#405)
**Changes:** Minimum define now always has the smallest value, and
maximum define always has the largest value.
**Required actions:** If these defines are used and enum values in
.proto file are not defined in ascending order, user code behaviour may
change. Check that user code doesn\'t expect the old, incorrect
first/last behaviour.
### Fix undefined behavior related to bool fields
**Rationale:** In C99, `bool` variables are not allowed to
have other values than `true` and `false`.
Compilers use this fact in optimization, and constructs like
`int foo = msg.has_field ? 100 : 0;` will give unexpected results
otherwise. Previously nanopb didn\'t enforce that decoded bool fields
had valid values.
**Changes:** Bool fields are now handled separately as
`PB_LTYPE_BOOL`. The `LTYPE` descriptor
numbers for other field types were renumbered.
**Required actions:** Source code files must be recompiled, but
regenerating `.pb.h`/`.pb.c` files from
`.proto` is not required. If user code directly uses the
nanopb internal field representation (search for
`PB_LTYPE_VARINT` in source), it may need updating.
Nanopb-0.3.9.1, 0.4.0 (2018-04-14)
----------------------------------
### Fix handling of string and bytes default values
**Rationale:** Previously nanopb didn't properly decode special
character escapes like `\200` emitted by protoc. This caused these
escapes to end up verbatim in the default values in .pb.c file.
**Changes:** Escapes are now decoded, and e.g. `\200` or `\x80`
results in {0x80} for bytes field and `"\x80"` for string field.
**Required actions:** If code has previously relied on `\` in default
value being passed through verbatim, it must now be changed to `\\`.
Nanopb-0.3.8 (2017-03-05)
-------------------------
### Fully drain substreams before closing
**Rationale:** If the substream functions were called directly and the
caller did not completely empty the substring before closing it, the
parent stream would be put into an incorrect state.
**Changes:** `pb_close_string_substream` can now error and returns a
boolean.
**Required actions:** Add error checking onto any call to
`pb_close_string_substream`.
### Change oneof format in .pb.c files
**Rationale:** Previously two oneofs in a single message would be
erroneously handled as part of the same union.
**Changes:** Oneofs fields now use special `PB_DATAOFFSET_UNION`
offset type in generated .pb.c files to distinguish whether they are the
first or following field inside an union.
**Required actions:** Regenerate `.pb.c/.pb.h` files with new nanopb
version if oneofs are used.
Nanopb-0.3.5 (2016-02-13)
-------------------------
### Add support for platforms without uint8_t
**Rationale:** Some platforms cannot access 8-bit sized values directly,
and do not define `uint8_t`. Nanopb previously didn\'t support these
platforms.
**Changes:** References to `uint8_t` were replaced with several
alternatives, one of them being a new `pb_byte_t` typedef. This in
turn uses `uint_least8_t` which means the smallest available type.
**Required actions:** If your platform does not have a
standards-compliant `stdint.h`, it may lack the definition for
`[u]int_least8_t`. This must be added manually, example can be found
in `extra/pb_syshdr.h`.
**Error indications:** Compiler error: `"unknown type name 'uint_least8_t'"`.
Nanopb-0.3.2 (2015-01-24)
-------------------------
### Add support for OneOfs
**Rationale:** Previously nanopb did not support the `oneof` construct
in `.proto` files. Those fields were generated as regular `optional`
fields.
**Changes:** OneOfs are now generated as C unions. Callback fields are
not supported inside oneof and generator gives an error.
**Required actions:** The generator option `no_unions` can be used to
restore old behaviour and to allow callbacks to be used. To use unions,
one change is needed: use `which_xxxx` field to detect which field is
present, instead of `has_xxxx`. Compare the value against
`MyStruct_myfield_tag`.
**Error indications:** Generator error: `"Callback fields inside of
oneof are not supported"`. Compiler error: `"Message"` has no member
named `"has_xxxx"`.
Nanopb-0.3.0 (2014-08-26)
-------------------------
### Separate field iterator logic to pb_common.c
**Rationale:** Originally, the field iteration logic was simple enough
to be duplicated in `pb_decode.c` and `pb_encode.c`. New field types
have made the logic more complex, which required the creation of a new
file to contain the common functionality.
**Changes:** There is a new file, `pb_common.c`, which must be included
in builds.
**Required actions:** Add `pb_common.c` to build rules. This file is
always required. Either `pb_decode.c` or `pb_encode.c` can still be
left out if some functionality is not needed.
**Error indications:** Linker error: undefined reference to
`pb_field_iter_begin`, `pb_field_iter_next` or similar.
### Change data type of field counts to pb_size_t
**Rationale:** Often nanopb is used with small arrays, such as 255 items
or less. Using a full `size_t` field to store the array count wastes
memory if there are many arrays. There already exists parameters
`PB_FIELD_16BIT` and `PB_FIELD_32BIT` which tell nanopb what is the
maximum size of arrays in use.
**Changes:** Generator will now use `pb_size_t` for the array
`_count` fields. The size of the type will be controlled by the
`PB_FIELD_16BIT` and `PB_FIELD_32BIT` compilation time options.
**Required actions:** Regenerate all `.pb.h` files. In some cases casts
to the `pb_size_t` type may need to be added in the user code when
accessing the `_count` fields.
**Error indications:** Incorrect data at runtime, crashes. But note that
other changes in the same version already require regenerating the files
and have better indications of errors, so this is only an issue for
development versions.
### Renamed some macros and identifiers
**Rationale:** Some names in nanopb core were badly chosen and
conflicted with ISO C99 reserved names or lacked a prefix. While they
haven\'t caused trouble so far, it is reasonable to switch to
non-conflicting names as these are rarely used from user code.
**Changes:** The following identifier names have changed:
- Macros:
- STATIC_ASSERT(x) -> PB_STATIC_ASSERT(x)
- UNUSED(x) -> PB_UNUSED(x)
- Include guards:
- PB_filename -> PB_filename_INCLUDED
- Structure forward declaration tags:
- _pb_field_t -> pb_field_s
- _pb_bytes_array_t -> pb_bytes_array_s
- _pb_callback_t -> pb_callback_s
- _pb_extension_type_t -> pb_extension_type_s
- _pb_extension_t -> pb_extension_s
- _pb_istream_t -> pb_istream_s
- _pb_ostream_t -> pb_ostream_s
**Required actions:** Regenerate all `.pb.c` files. If you use any of
the above identifiers in your application code, perform search-replace
to the new name.
**Error indications:** Compiler errors on lines with the macro/type
names.
Nanopb-0.2.9 (2014-08-09)
-------------------------
### Change semantics of generator -e option
**Rationale:** Some compilers do not accept filenames with two dots
(like in default extension .pb.c). The `-e` option to the generator
allowed changing the extension, but not skipping the extra dot.
**Changes:** The `-e` option in generator will no longer add the
prepending dot. The default value has been adjusted accordingly to
`.pb.c` to keep the default behaviour the same as before.
**Required actions:** Only if using the generator -e option. Add dot
before the parameter value on the command line.
**Error indications:** File not found when trying to compile generated
files.
Nanopb-0.2.7 (2014-04-07)
-------------------------
### Changed pointer-type bytes field datatype
**Rationale:** In the initial pointer encoding support since
nanopb-0.2.5, the bytes type used a separate `pb_bytes_ptr_t` type to
represent `bytes` fields. This made it easy to encode data from a
separate, user-allocated buffer. However, it made the internal logic
more complex and was inconsistent with the other types.
**Changes:** Dynamically allocated bytes fields now have the
`pb_bytes_array_t` type, just like statically allocated ones.
**Required actions:** Only if using pointer-type fields with the bytes
datatype. Change any access to `msg->field.size` to
`msg->field->size`. Change any allocation to reserve space of amount
`PB_BYTES_ARRAY_T_ALLOCSIZE(n)`. If the data pointer was begin
assigned from external source, implement the field using a callback
function instead.
**Error indications:** Compiler error: unknown type name
`pb_bytes_ptr_t`.
Nanopb-0.2.4 (2013-11-07)
-------------------------
### Remove the NANOPB_INTERNALS compilation option
**Rationale:** Having the option in the headers required the functions
to be non-static, even if the option is not used. This caused errors on
some static analysis tools.
**Changes:** The `\#ifdef` and associated functions were removed from
the header.
**Required actions:** Only if the `NANOPB_INTERNALS` option was
previously used. Actions are as listed under nanopb-0.1.3 and
nanopb-0.1.6.
**Error indications:** Compiler warning: implicit declaration of
function `pb_dec_string`, `pb_enc_string`, or similar.
Nanopb-0.2.1 (2013-04-14)
-------------------------
### Callback function signature
**Rationale:** Previously the auxiliary data to field callbacks was
passed as `void*`. This allowed passing of any data, but made it
unnecessarily complex to return a pointer from callback.
**Changes:** The callback function parameter was changed to `void**`.
**Required actions:** You can continue using the old callback style by
defining `PB_OLD_CALLBACK_STYLE`. Recommended action is to:
- Change the callback signatures to contain `void**` for decoders and `void * const *` for encoders.
- Change the callback function body to use **arg` instead of `arg`.
**Error indications:** Compiler warning: assignment from incompatible
pointer type, when initializing `funcs.encode` or `funcs.decode`.
Nanopb-0.2.0 (2013-03-02)
-------------------------
### Reformatted generated .pb.c file using macros
**Rationale:** Previously the generator made a list of C `pb_field_t`
initializers in the .pb.c file. This led to a need to regenerate all
.pb.c files after even small changes to the `pb_field_t` definition.
**Changes:** Macros were added to pb.h which allow for cleaner
definition of the .pb.c contents. By changing the macro definitions,
changes to the field structure are possible without breaking
compatibility with old .pb.c files.
**Required actions:** Regenerate all .pb.c files from the .proto
sources.
**Error indications:** Compiler warning: implicit declaration of
function `pb_delta_end`.
### Changed pb_type_t definitions
**Rationale:** The `pb_type_t` was previously an enumeration type.
This caused warnings on some compilers when using bitwise operations to
set flags inside the values.
**Changes:** The `pb_type_t` was changed to *typedef uint8_t*. The
values were changed to `#define`. Some value names were changed for
consistency.
**Required actions:** Only if you directly access the
`pb_field_t` contents in your own code, something which is
not usually done. Needed changes:
- Change `PB_HTYPE_ARRAY` to `PB_HTYPE_REPEATED`.
- Change `PB_HTYPE_CALLBACK` to `PB_ATYPE()` and `PB_ATYPE_CALLBACK`.
**Error indications:** Compiler error: `PB_HTYPE_ARRAY` or
`PB_HTYPE_CALLBACK` undeclared.
Nanopb-0.1.6 (2012-09-02)
-------------------------
### Refactored field decoder interface
**Rationale:** Similarly to field encoders in nanopb-0.1.3.
**Changes:** New functions with names `pb_decode_*` were added.
**Required actions:** By defining NANOPB_INTERNALS, you can still keep
using the old functions. Recommended action is to replace any calls with
the newer `pb_decode_*` equivalents.
**Error indications:** Compiler warning: implicit declaration of
function `pb_dec_string`, `pb_dec_varint`, `pb_dec_submessage` or
similar.
Nanopb-0.1.3 (2012-06-12)
-------------------------
### Refactored field encoder interface
**Rationale:** The old `pb_enc_*` functions were designed mostly for
the internal use by the core. Because they are internally accessed
through function pointers, their signatures had to be common. This led
to a confusing interface for external users.
**Changes:** New functions with names `pb_encode_*` were added. These
have easier to use interfaces. The old functions are now only thin
wrappers for the new interface.
**Required actions:** By defining NANOPB_INTERNALS, you can still keep
using the old functions. Recommended action is to replace any calls with
the newer `pb_encode_*` equivalents.
**Error indications:** Compiler warning: implicit declaration of
function `pb_enc_string`, *pb_enc_varint,`pb_enc_submessage\` or
similar.

1047
lib/nanopb/docs/reference.md Normal file
View File

File diff suppressed because it is too large Load Diff

92
lib/nanopb/docs/security.md Normal file
View File

@@ -0,0 +1,92 @@
# Nanopb: Security model
Importance of security in a Protocol Buffers library
----------------------------------------------------
In the context of protocol buffers, security comes into play when
decoding untrusted data. Naturally, if the attacker can modify the
contents of a protocol buffers message, he can feed the application any
values possible. Therefore the application itself must be prepared to
receive untrusted values.
Where nanopb plays a part is preventing the attacker from running
arbitrary code on the target system. Mostly this means that there must
not be any possibility to cause buffer overruns, memory corruption or
invalid pointers by the means of crafting a malicious message.
Division of trusted and untrusted data
--------------------------------------
The following data is regarded as **trusted**. It must be under the
control of the application writer. Malicious data in these structures
could cause security issues, such as execution of arbitrary code:
1. Callback, pointer and extension fields in message structures given
to pb_encode() and pb_decode(). These fields are memory pointers,
and are generated depending on the message definition in the .proto
file.
2. The automatically generated field definitions, i.e.
`pb_msgdesc_t`.
3. Contents of the `pb_istream_t` and `pb_ostream_t` structures
(this does not mean the contents of the stream itself, just the
stream definition).
The following data is regarded as **untrusted**. Invalid/malicious data
in these will cause "garbage in, garbage out" behaviour. It will not
cause buffer overflows, information disclosure or other security
problems:
1. All data read from `pb_istream_t`.
2. All fields in message structures, except:
- callbacks (`pb_callback_t` structures)
- pointer fields and `_count` fields for pointers
- extensions (`pb_extension_t` structures)
Invariants
----------
The following invariants are maintained during operation, even if the
untrusted data has been maliciously crafted:
1. Nanopb will never read more than `bytes_left` bytes from
`pb_istream_t`.
2. Nanopb will never write more than `max_size` bytes to
`pb_ostream_t`.
3. Nanopb will never access memory out of bounds of the message
structure.
4. After `pb_decode()` returns successfully, the message structure will
be internally consistent:
- The `count` fields of arrays will not exceed the array size.
- The `size` field of bytes will not exceed the allocated size.
- All string fields will have null terminator.
- bool fields will have valid true/false values (since
nanopb-0.3.9.4)
- pointer fields will be either `NULL` or point to valid data
5. After `pb_encode()` returns successfully, the resulting message is a
valid protocol buffers message. (Except if user-defined callbacks
write incorrect data.)
6. All memory allocated by `pb_decode()` will be released by a subsequent
call to `pb_release()` on the same message.
Further considerations
----------------------
Even if the nanopb library is free of any security issues, there are
still several possible attack vectors that the application author must
consider. The following list is not comprehensive:
1. Stack usage may depend on the contents of the message. The message
definition places an upper bound on how much stack will be used.
Tests should be run with all fields present, to record the maximum
possible stack usage.
2. Callbacks can do anything. The code for the callbacks must be
carefully checked if they are used with untrusted data.
3. If using stream input, a maximum size should be set in
`pb_istream_t` to stop a denial of service attack from using an
infinite message.
4. If using network sockets as streams, a timeout should be set to stop
denial of service attacks.
5. If using `malloc()` support, some method of limiting memory use
should be employed. This can be done by defining custom
`pb_realloc()` function. Nanopb will properly detect and handle
failed memory allocations.

173
lib/nanopb/docs/whats_new.md Normal file
View File

@@ -0,0 +1,173 @@
# Nanopb: New features in nanopb 0.4
## What's new in nanopb 0.4
Long in the making, nanopb 0.4 has seen some wide reaching improvements
in reaction to the development of the rest of the protobuf ecosystem.
This document showcases features that are not immediately visible, but
that you may want to take advantage of.
A lot of effort has been spent in retaining backwards and forwards
compatibility with previous nanopb versions. For a list of breaking
changes, see [migration document](migration.html)
### New field descriptor format
The basic design of nanopb has always been that the information about
messages is stored in a compact descriptor format, which is iterated in
runtime. Initially it was very tightly tied with encoder and decoder
logic.
In nanopb-0.3.0 the field iteration logic was separated to
`pb_common.c`. Already at that point it was clear that the old format
was getting too limited, but it wasn't extended at that time.
Now in 0.4, the descriptor format was completely decoupled from the
encoder and decoder logic, and redesigned to meet new demands.
Previously each field was stored as `pb_field_t` struct, which was
between 8 and 32 bytes in size, depending on compilation options and
platform. Now information about fields is stored as a variable length
sequence of `uint32_t` data words. There are 1, 2, 4 and 8 word formats,
with the 8 word format containing plenty of space for future
extensibility.
One benefit of the variable length format is that most messages now take
less storage space. Most fields use 2 words, while simple fields in
small messages require only 1 word. Benefit is larger if code previously
required `PB_FIELD_16BIT` or `PB_FIELD_32BIT` options. In
the `AllTypes` test case, 0.3 had data size of 1008 bytes in
8-bit configuration and 1408 bytes in 16-bit configuration. New format
in 0.4 takes 896 bytes for either of these.
In addition, the new decoupling has allowed moving most of the field
descriptor data into FLASH on Harvard architectures, such as AVR.
Previously nanopb was quite RAM-heavy on AVR, which cannot put normal
constants in flash like most other platforms do.
### Python packaging for generator
Nanopb generator is now available as a Python package, installable using
`pip` package manager. This will reduce the need for binary
packages, as if you have Python already installed you can just
`pip install nanopb` and have the generator available on path as
`nanopb_generator`.
The generator can also take advantage of the Python-based `protoc`
available in `grpcio-tools` Python package. If you also install that,
there is no longer a need to have binary `protoc` available.
### Generator now automatically calls protoc
Initially, nanopb generator was used in two steps: first calling
`protoc` to parse the `.proto` file into `.pb` binary
format, and then calling `nanopb_generator.py` to output the
`.pb.h` and `.pb.c` files.
Nanopb 0.2.3 added support for running as a `protoc` plugin, which
allowed single-step generation using `--nanopb_out` parameter. However,
the plugin mode has two complications: passing options to nanopb
generator itself becomes more difficult, and the generator does not know
the actual path of input files. The second limitation has been
particularly problematic for locating `.options` files.
Both of these older methods still work and will remain supported.
However, now `nanopb_generator` can also take `.proto` files
directly and it will transparently call `protoc` in the background.
### Callbacks bound by function name
Since its very beginnings, nanopb has supported field callbacks to allow
processing structures that are larger than what could fit in memory at
once. So far the callback functions have been stored in the message
structure in a `pb_callback_t` struct.
Storing pointers along with user data is somewhat risky from a security
point of view. In addition it has caused problems with `oneof` fields,
which reuse the same storage space for multiple submessages. Because
there is no separate area for each submessage, there is no space to
store the callback pointers either.
Nanopb-0.4.0 introduces callbacks that are referenced by the function
name instead of setting the pointers separately. This should work well
for most applications that have a single callback function for each
message type. For more complex needs, `pb_callback_t` will also remain
supported.
Function name callbacks also allow specifying custom data types for
inclusion in the message structure. For example, you could have
`MyObject*` pointer along with other message fields, and then process
that object in custom way in your callback.
This feature is demonstrated in
[tests/oneof_callback](https://github.com/nanopb/nanopb/tree/master/tests/oneof_callback) test case and
[examples/network_server](https://github.com/nanopb/nanopb/tree/master/examples/network_server) example.
### Message level callback for oneofs
As mentioned above, callbacks inside submessages inside oneofs have been
problematic to use. To make using `pb_callback_t`-style callbacks there
possible, a new generator option `submsg_callback` was added.
Setting this option to true will cause a new message level callback to
be added before the `which_field` of the oneof. This callback will be
called when the submessage tag number is known, but before the actual
message is decoded. The callback can either choose to set callback
pointers inside the submessage, or just completely decode the submessage
there and then. If any unread data remains after the callback returns,
normal submessage decoding will continue.
There is an example of this in [tests/oneof_callback](https://github.com/nanopb/nanopb/tree/master/tests/oneof_callback) test case.
### Binding message types to custom structures
It is often said that good C code is chock full of macros. Or maybe I
got it wrong. But since nanopb 0.2, the field descriptor generation has
heavily relied on macros. This allows it to automatically adapt to
differences in type alignment on different platforms, and to decouple
the Python generation logic from how the message descriptors are
implemented on the C side.
Now in 0.4.0, I've made the macros even more abstract. Time will tell
whether this was such a great idea that I think it is, but now the
complete list of fields in each message is available in `.pb.h` file.
This allows a kind of metaprogramming using [X-macros]()
One feature that this can be used for is binding the message descriptor
to a custom structure or C++ class type. You could have a bunch of other
fields in the structure and even the datatypes can be different to an
extent, and nanopb will automatically detect the size and position of
each field. The generated `.pb.c` files now just have calls of
`PB_BIND(msgname, structname, width)`. Adding a similar
call to your own code will bind the message to your own structure.
### UTF-8 validation
Protobuf format defines that strings should consist of valid UTF-8
codepoints. Previously nanopb has not enforced this, requiring extra
care in the user code. Now optional UTF-8 validation is available with
compilation option `PB_VALIDATE_UTF8`.
### Double to float conversion
Some platforms such as `AVR` do not support the `double`
datatype, instead making it an alias for `float`. This has resulted in
problems when trying to process message types containing `double` fields
generated on other machines. There has been an example on how to
manually perform the conversion between `double` and
`float`.
Now that example is integrated as an optional feature in nanopb core. By
defining `PB_CONVERT_DOUBLE_FLOAT`, the required conversion between 32-
and 64-bit floating point formats happens automatically on decoding and
encoding.
### Improved testing
Testing on embedded platforms has been integrated in the continuous
testing environment. Now all of the 80+ test cases are automatically run
on STM32 and AVR targets. Previously only a few specialized test cases
were manually tested on embedded systems.
Nanopb fuzzer has also been integrated in Google's [OSSFuzz](https://google.github.io/oss-fuzz/)
platform, giving a huge boost in the CPU power available for randomized
testing.

15
lib/nanopb/examples/cmake_relpath/CMakeLists.txt Normal file
View File

@@ -0,0 +1,15 @@
cmake_minimum_required(VERSION 2.8)
project(NANOPB_CMAKE_SIMPLE C)
set(CMAKE_MODULE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/../../extra)
find_package(Nanopb REQUIRED)
include_directories(${NANOPB_INCLUDE_DIRS})
nanopb_generate_cpp(PROTO_SRCS PROTO_HDRS RELPATH proto
proto/simple.proto proto/sub/unlucky.proto)
include_directories(${CMAKE_CURRENT_BINARY_DIR})
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Wall -Werror -g -O0")
add_executable(simple simple.c ${PROTO_SRCS} ${PROTO_HDRS})

18
lib/nanopb/examples/cmake_relpath/README.txt Normal file
View File

@@ -0,0 +1,18 @@
Nanopb example "simple" using CMake
=======================
This example is the same as the simple nanopb example but built using CMake.
Example usage
-------------
On Linux, create a build directory and then call cmake:
nanopb/examples/cmake_simple$ mkdir build
nanopb/examples/cmake_simple$ cd build/
nanopb/examples/cmake_simple/build$ cmake ..
nanopb/examples/cmake_simple/build$ make
After that, you can run it with the command: ./simple
On other platforms supported by CMake, refer to CMake instructions.

11
lib/nanopb/examples/cmake_relpath/proto/simple.proto Normal file
View File

@@ -0,0 +1,11 @@
// A very simple protocol definition, consisting of only
// one message.
syntax = "proto2";
import "sub/unlucky.proto";
message SimpleMessage {
required int32 lucky_number = 1;
required UnluckyNumber unlucky = 2;
}

5
lib/nanopb/examples/cmake_relpath/proto/sub/unlucky.proto Normal file
View File

@@ -0,0 +1,5 @@
syntax = "proto2";
message UnluckyNumber {
required uint32 number = 1;
}

73
lib/nanopb/examples/cmake_relpath/simple.c Normal file
View File

@@ -0,0 +1,73 @@
#include <stdio.h>
#include <pb_encode.h>
#include <pb_decode.h>
#include "simple.pb.h"
int main()
{
/* This is the buffer where we will store our message. */
uint8_t buffer[128];
size_t message_length;
bool status;
/* Encode our message */
{
/* Allocate space on the stack to store the message data.
*
* Nanopb generates simple struct definitions for all the messages.
* - check out the contents of simple.pb.h!
* It is a good idea to always initialize your structures
* so that you do not have garbage data from RAM in there.
*/
SimpleMessage message = SimpleMessage_init_zero;
/* Create a stream that will write to our buffer. */
pb_ostream_t stream = pb_ostream_from_buffer(buffer, sizeof(buffer));
/* Fill in the lucky number */
message.lucky_number = 13;
message.unlucky.number = 42;
/* Now we are ready to encode the message! */
status = pb_encode(&stream, SimpleMessage_fields, &message);
message_length = stream.bytes_written;
/* Then just check for any errors.. */
if (!status)
{
printf("Encoding failed: %s\n", PB_GET_ERROR(&stream));
return 1;
}
}
/* Now we could transmit the message over network, store it in a file or
* wrap it to a pigeon's leg.
*/
/* But because we are lazy, we will just decode it immediately. */
{
/* Allocate space for the decoded message. */
SimpleMessage message = SimpleMessage_init_zero;
/* Create a stream that reads from the buffer. */
pb_istream_t stream = pb_istream_from_buffer(buffer, message_length);
/* Now we are ready to decode the message. */
status = pb_decode(&stream, SimpleMessage_fields, &message);
/* Check for errors... */
if (!status)
{
printf("Decoding failed: %s\n", PB_GET_ERROR(&stream));
return 1;
}
/* Print the data contained in the message. */
printf("Your lucky number was %d!\n", message.lucky_number);
printf("Your unlucky number was %u!\n", message.unlucky.number);
}
return 0;
}

13
lib/nanopb/examples/cmake_simple/CMakeLists.txt Normal file
View File

@@ -0,0 +1,13 @@
cmake_minimum_required(VERSION 2.8)
project(NANOPB_CMAKE_SIMPLE C)
set(CMAKE_MODULE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/../../extra)
find_package(Nanopb REQUIRED)
include_directories(${NANOPB_INCLUDE_DIRS})
nanopb_generate_cpp(PROTO_SRCS PROTO_HDRS simple.proto)
include_directories(${CMAKE_CURRENT_BINARY_DIR})
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Wall -Werror -g -O0")
add_executable(simple simple.c ${PROTO_SRCS} ${PROTO_HDRS})

18
lib/nanopb/examples/cmake_simple/README.txt Normal file
View File

@@ -0,0 +1,18 @@
Nanopb example "simple" using CMake
=======================
This example is the same as the simple nanopb example but built using CMake.
Example usage
-------------
On Linux, create a build directory and then call cmake:
nanopb/examples/cmake_simple$ mkdir build
nanopb/examples/cmake_simple$ cd build/
nanopb/examples/cmake_simple/build$ cmake ..
nanopb/examples/cmake_simple/build$ make
After that, you can run it with the command: ./simple
On other platforms supported by CMake, refer to CMake instructions.

71
lib/nanopb/examples/cmake_simple/simple.c Normal file
View File

@@ -0,0 +1,71 @@
#include <stdio.h>
#include <pb_encode.h>
#include <pb_decode.h>
#include "simple.pb.h"
int main()
{
/* This is the buffer where we will store our message. */
uint8_t buffer[128];
size_t message_length;
bool status;
/* Encode our message */
{
/* Allocate space on the stack to store the message data.
*
* Nanopb generates simple struct definitions for all the messages.
* - check out the contents of simple.pb.h!
* It is a good idea to always initialize your structures
* so that you do not have garbage data from RAM in there.
*/
SimpleMessage message = SimpleMessage_init_zero;
/* Create a stream that will write to our buffer. */
pb_ostream_t stream = pb_ostream_from_buffer(buffer, sizeof(buffer));
/* Fill in the lucky number */
message.lucky_number = 13;
/* Now we are ready to encode the message! */
status = pb_encode(&stream, SimpleMessage_fields, &message);
message_length = stream.bytes_written;
/* Then just check for any errors.. */
if (!status)
{
printf("Encoding failed: %s\n", PB_GET_ERROR(&stream));
return 1;
}
}
/* Now we could transmit the message over network, store it in a file or
* wrap it to a pigeon's leg.
*/
/* But because we are lazy, we will just decode it immediately. */
{
/* Allocate space for the decoded message. */
SimpleMessage message = SimpleMessage_init_zero;
/* Create a stream that reads from the buffer. */
pb_istream_t stream = pb_istream_from_buffer(buffer, message_length);
/* Now we are ready to decode the message. */
status = pb_decode(&stream, SimpleMessage_fields, &message);
/* Check for errors... */
if (!status)
{
printf("Decoding failed: %s\n", PB_GET_ERROR(&stream));
return 1;
}
/* Print the data contained in the message. */
printf("Your lucky number was %d!\n", message.lucky_number);
}
return 0;
}

9
lib/nanopb/examples/cmake_simple/simple.proto Normal file
View File

@@ -0,0 +1,9 @@
// A very simple protocol definition, consisting of only
// one message.
syntax = "proto2";
message SimpleMessage {
required int32 lucky_number = 1;
}

1
lib/nanopb/examples/conan_dependency/.gitignore vendored Normal file
View File

@@ -0,0 +1 @@
build/

13
lib/nanopb/examples/conan_dependency/CMakeLists.txt Normal file
View File

@@ -0,0 +1,13 @@
cmake_minimum_required(VERSION 3.20)
project(simple C)
include(${CMAKE_BINARY_DIR}/conanbuildinfo.cmake)
conan_basic_setup()
add_library(simple-protos STATIC
${CMAKE_BINARY_DIR}/src/simple.pb.c
)
add_executable(simple ${CMAKE_BINARY_DIR}/src/simple.c)
target_link_libraries(simple ${CONAN_LIBS} simple-protos)

28
lib/nanopb/examples/conan_dependency/README.md Normal file
View File

@@ -0,0 +1,28 @@
# About
This example shows how to use Conan to pull in the header files and static libraries
for `nanopb` and incorporate them into a very simple CMake application.
## How To Run
### Before using this example
The `conanfile.py` here imports `0.4.6` for `nanopb` and uses the packaged artifacts
to build a simple application. You'll likely need to build this yourself, so
checkout the tagged version and run `conan create .` in the base of this repository
### Running line by line
To run though the build one step at a time, use the following commands.
```sh
mkdir build
cd build
conan install ..
conan source ..
conan build ..
conan package ..
```
The `conanfile.py` has been commented to explain the workflow
### Installing to cache
To have everything build at once and install to your local Conan cache
```sh
conan create .
```

42
lib/nanopb/examples/conan_dependency/conanfile.py Normal file
View File

@@ -0,0 +1,42 @@
from conans import ConanFile, CMake
class SimpleProtosConan(ConanFile):
name = "simple_protos"
version = "1.0.0"
description = "An example of importing nanopb as a conan artifact"
settings = "os", "compiler", "build_type", "arch"
generators = "cmake"
exports = "*"
def requirements(self):
self.requires("nanopb/0.4.6")
def imports(self):
# Includes the nanopb headers
self.copy("*.h")
# Includes the compiled nanopb libraries
self.copy("*", src="lib", dst="lib")
# Includes the protoc plugin
self.copy("*", src="bin", dst="bin")
# Includes the python libraries that `bin` reaches out to
self.copy("*", src="local", dst="local")
def source(self):
# To include the packages from nanopb, we need to get their path in cache
nanopb_package_root = self.deps_cpp_info["nanopb"].rootpath
python_path=f"PYTHONPATH={nanopb_package_root}/local/lib/python3.10/dist-packages"
plugin=f"--plugin={nanopb_package_root}/bin/protoc-gen-nanopb"
# These next values grab this environments source and proto directories
output=f"--nanopb_out={self.source_folder}/src"
proto_flags=f"-I {self.source_folder}/protos simple.proto"
self.run(f"{python_path} protoc {plugin} {output} {proto_flags}")
def build(self):
cmake = CMake(self)
cmake.configure()
cmake.build()
def package(self):
self.copy("simple", dst="bin", src="bin")

9
lib/nanopb/examples/conan_dependency/protos/simple.proto Normal file
View File

@@ -0,0 +1,9 @@
// A very simple protocol definition, consisting of only
// one message.
syntax = "proto2";
message SimpleMessage {
required int32 lucky_number = 1;
}

71
lib/nanopb/examples/conan_dependency/src/simple.c Normal file
View File

@@ -0,0 +1,71 @@
#include <stdio.h>
#include <pb_encode.h>
#include <pb_decode.h>
#include "simple.pb.h"
int main()
{
/* This is the buffer where we will store our message. */
uint8_t buffer[128];
size_t message_length;
bool status;
/* Encode our message */
{
/* Allocate space on the stack to store the message data.
*
* Nanopb generates simple struct definitions for all the messages.
* - check out the contents of simple.pb.h!
* It is a good idea to always initialize your structures
* so that you do not have garbage data from RAM in there.
*/
SimpleMessage message = SimpleMessage_init_zero;
/* Create a stream that will write to our buffer. */
pb_ostream_t stream = pb_ostream_from_buffer(buffer, sizeof(buffer));
/* Fill in the lucky number */
message.lucky_number = 13;
/* Now we are ready to encode the message! */
status = pb_encode(&stream, SimpleMessage_fields, &message);
message_length = stream.bytes_written;
/* Then just check for any errors.. */
if (!status)
{
printf("Encoding failed: %s\n", PB_GET_ERROR(&stream));
return 1;
}
}
/* Now we could transmit the message over network, store it in a file or
* wrap it to a pigeon's leg.
*/
/* But because we are lazy, we will just decode it immediately. */
{
/* Allocate space for the decoded message. */
SimpleMessage message = SimpleMessage_init_zero;
/* Create a stream that reads from the buffer. */
pb_istream_t stream = pb_istream_from_buffer(buffer, message_length);
/* Now we are ready to decode the message. */
status = pb_decode(&stream, SimpleMessage_fields, &message);
/* Check for errors... */
if (!status)
{
printf("Decoding failed: %s\n", PB_GET_ERROR(&stream));
return 1;
}
/* Print the data contained in the message. */
printf("Your lucky number was %d!\n", message.lucky_number);
}
return 0;
}

17
lib/nanopb/examples/network_server/Makefile Normal file
View File

@@ -0,0 +1,17 @@
# Include the nanopb provided Makefile rules
include ../../extra/nanopb.mk
# Compiler flags to enable all warnings & debug info
CFLAGS = -ansi -Wall -Werror -g -O0
CFLAGS += -I$(NANOPB_DIR)
all: server client
.SUFFIXES:
clean:
rm -f server client fileproto.pb.c fileproto.pb.h
%: %.c common.c fileproto.pb.c
$(CC) $(CFLAGS) -o $@ $^ $(NANOPB_CORE)

60
lib/nanopb/examples/network_server/README.txt Normal file
View File

@@ -0,0 +1,60 @@
Nanopb example "network_server"
===============================
This example demonstrates the use of nanopb to communicate over network
connections. It consists of a server that sends file listings, and of
a client that requests the file list from the server.
Example usage
-------------
user@host:~/nanopb/examples/network_server$ make # Build the example
protoc -ofileproto.pb fileproto.proto
python ../../generator/nanopb_generator.py fileproto.pb
Writing to fileproto.pb.h and fileproto.pb.c
cc -ansi -Wall -Werror -I .. -g -O0 -I../.. -o server server.c
../../pb_decode.c ../../pb_encode.c fileproto.pb.c common.c
cc -ansi -Wall -Werror -I .. -g -O0 -I../.. -o client client.c
../../pb_decode.c ../../pb_encode.c fileproto.pb.c common.c
user@host:~/nanopb/examples/network_server$ ./server & # Start the server on background
[1] 24462
petteri@oddish:~/nanopb/examples/network_server$ ./client /bin # Request the server to list /bin
Got connection.
Listing directory: /bin
1327119 bzdiff
1327126 bzless
1327147 ps
1327178 ntfsmove
1327271 mv
1327187 mount
1327259 false
1327266 tempfile
1327285 zfgrep
1327165 gzexe
1327204 nc.openbsd
1327260 uname
Details of implementation
-------------------------
fileproto.proto contains the portable Google Protocol Buffers protocol definition.
It could be used as-is to implement a server or a client in any other language, for
example Python or Java.
fileproto.options contains the nanopb-specific options for the protocol file. This
sets the amount of space allocated for file names when decoding messages.
common.c/h contains functions that allow nanopb to read and write directly from
network socket. This way there is no need to allocate a separate buffer to store
the message.
server.c contains the code to open a listening socket, to respond to clients and
to list directory contents.
client.c contains the code to connect to a server, to send a request and to print
the response message.
The code is implemented using the POSIX socket api, but it should be easy enough
to port into any other socket api, such as lwip.

138
lib/nanopb/examples/network_server/client.c Normal file
View File

@@ -0,0 +1,138 @@
/* This is a simple TCP client that connects to port 1234 and prints a list
* of files in a given directory.
*
* It directly deserializes and serializes messages from network, minimizing
* memory use.
*
* For flexibility, this example is implemented using posix api.
* In a real embedded system you would typically use some other kind of
* a communication and filesystem layer.
*/
#include <sys/socket.h>
#include <sys/types.h>
#include <netinet/in.h>
#include <unistd.h>
#include <dirent.h>
#include <stdio.h>
#include <string.h>
#include <pb_encode.h>
#include <pb_decode.h>
#include "fileproto.pb.h"
#include "common.h"
/* This callback function will be called once for each filename received
* from the server. The filenames will be printed out immediately, so that
* no memory has to be allocated for them.
*/
bool ListFilesResponse_callback(pb_istream_t *istream, pb_ostream_t *ostream, const pb_field_iter_t *field)
{
PB_UNUSED(ostream);
if (istream != NULL && field->tag == ListFilesResponse_file_tag)
{
FileInfo fileinfo = {};
if (!pb_decode(istream, FileInfo_fields, &fileinfo))
return false;
printf("%-10lld %s\n", (long long)fileinfo.inode, fileinfo.name);
}
return true;
}
/* This function sends a request to socket 'fd' to list the files in
* directory given in 'path'. The results received from server will
* be printed to stdout.
*/
bool listdir(int fd, char *path)
{
/* Construct and send the request to server */
{
ListFilesRequest request = {};
pb_ostream_t output = pb_ostream_from_socket(fd);
/* In our protocol, path is optional. If it is not given,
* the server will list the root directory. */
if (path == NULL)
{
request.has_path = false;
}
else
{
request.has_path = true;
if (strlen(path) + 1 > sizeof(request.path))
{
fprintf(stderr, "Too long path.\n");
return false;
}
strcpy(request.path, path);
}
/* Encode the request. It is written to the socket immediately
* through our custom stream. */
if (!pb_encode_delimited(&output, ListFilesRequest_fields, &request))
{
fprintf(stderr, "Encoding failed: %s\n", PB_GET_ERROR(&output));
return false;
}
}
/* Read back the response from server */
{
ListFilesResponse response = {};
pb_istream_t input = pb_istream_from_socket(fd);
if (!pb_decode_delimited(&input, ListFilesResponse_fields, &response))
{
fprintf(stderr, "Decode failed: %s\n", PB_GET_ERROR(&input));
return false;
}
/* If the message from server decodes properly, but directory was
* not found on server side, we get path_error == true. */
if (response.path_error)
{
fprintf(stderr, "Server reported error.\n");
return false;
}
}
return true;
}
int main(int argc, char **argv)
{
int sockfd;
struct sockaddr_in servaddr;
char *path = NULL;
if (argc > 1)
path = argv[1];
sockfd = socket(AF_INET, SOCK_STREAM, 0);
/* Connect to server running on localhost:1234 */
memset(&servaddr, 0, sizeof(servaddr));
servaddr.sin_family = AF_INET;
servaddr.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
servaddr.sin_port = htons(1234);
if (connect(sockfd, (struct sockaddr *)&servaddr, sizeof(servaddr)) != 0)
{
perror("connect");
return 1;
}
/* Send the directory listing request */
if (!listdir(sockfd, path))
return 2;
/* Close connection */
close(sockfd);
return 0;
}

43
lib/nanopb/examples/network_server/common.c Normal file
View File

@@ -0,0 +1,43 @@
/* Simple binding of nanopb streams to TCP sockets.
*/
#include <sys/socket.h>
#include <sys/types.h>
#include <pb_encode.h>
#include <pb_decode.h>
#include "common.h"
static bool write_callback(pb_ostream_t *stream, const uint8_t *buf, size_t count)
{
int fd = (intptr_t)stream->state;
return send(fd, buf, count, 0) == count;
}
static bool read_callback(pb_istream_t *stream, uint8_t *buf, size_t count)
{
int fd = (intptr_t)stream->state;
int result;
if (count == 0)
return true;
result = recv(fd, buf, count, MSG_WAITALL);
if (result == 0)
stream->bytes_left = 0; /* EOF */
return result == count;
}
pb_ostream_t pb_ostream_from_socket(int fd)
{
pb_ostream_t stream = {&write_callback, (void*)(intptr_t)fd, SIZE_MAX, 0};
return stream;
}
pb_istream_t pb_istream_from_socket(int fd)
{
pb_istream_t stream = {&read_callback, (void*)(intptr_t)fd, SIZE_MAX};
return stream;
}

9
lib/nanopb/examples/network_server/common.h Normal file
View File

@@ -0,0 +1,9 @@
#ifndef _PB_EXAMPLE_COMMON_H_
#define _PB_EXAMPLE_COMMON_H_
#include <pb.h>
pb_ostream_t pb_ostream_from_socket(int fd);
pb_istream_t pb_istream_from_socket(int fd);
#endif

16
lib/nanopb/examples/network_server/fileproto.options Normal file
View File

@@ -0,0 +1,16 @@
# This file defines the nanopb-specific options for the messages defined
# in fileproto.proto.
#
# If you come from high-level programming background, the hardcoded
# maximum lengths may disgust you. However, if your microcontroller only
# has a few kB of ram to begin with, setting reasonable limits for
# filenames is ok.
#
# On the other hand, using the callback interface, it is not necessary
# to set a limit on the number of files in the response.
* include:"sys/types.h"
* include:"dirent.h"
ListFilesResponse.file type:FT_CALLBACK, callback_datatype:"DIR*"
ListFilesRequest.path max_size:128
FileInfo.name max_size:128

20
lib/nanopb/examples/network_server/fileproto.proto Normal file
View File

@@ -0,0 +1,20 @@
// This defines protocol for a simple server that lists files.
//
// See also the nanopb-specific options in fileproto.options.
syntax = "proto2";
message ListFilesRequest {
optional string path = 1 [default = "/"];
}
message FileInfo {
required uint64 inode = 1;
required string name = 2;
}
message ListFilesResponse {
optional bool path_error = 1 [default = false];
repeated FileInfo file = 2;
}

164
lib/nanopb/examples/network_server/server.c Normal file
View File

@@ -0,0 +1,164 @@
/* This is a simple TCP server that listens on port 1234 and provides lists
* of files to clients, using a protocol defined in file_server.proto.
*
* It directly deserializes and serializes messages from network, minimizing
* memory use.
*
* For flexibility, this example is implemented using posix api.
* In a real embedded system you would typically use some other kind of
* a communication and filesystem layer.
*/
#include <sys/socket.h>
#include <sys/types.h>
#include <netinet/in.h>
#include <unistd.h>
#include <dirent.h>
#include <stdio.h>
#include <string.h>
#include <pb_encode.h>
#include <pb_decode.h>
#include "fileproto.pb.h"
#include "common.h"
/* This callback function will be called during the encoding.
* It will write out any number of FileInfo entries, without consuming unnecessary memory.
* This is accomplished by fetching the filenames one at a time and encoding them
* immediately.
*/
bool ListFilesResponse_callback(pb_istream_t *istream, pb_ostream_t *ostream, const pb_field_iter_t *field)
{
PB_UNUSED(istream);
if (ostream != NULL && field->tag == ListFilesResponse_file_tag)
{
DIR *dir = *(DIR**)field->pData;
struct dirent *file;
FileInfo fileinfo = {};
while ((file = readdir(dir)) != NULL)
{
fileinfo.inode = file->d_ino;
strncpy(fileinfo.name, file->d_name, sizeof(fileinfo.name));
fileinfo.name[sizeof(fileinfo.name) - 1] = '\0';
/* This encodes the header for the field, based on the constant info
* from pb_field_t. */
if (!pb_encode_tag_for_field(ostream, field))
return false;
/* This encodes the data for the field, based on our FileInfo structure. */
if (!pb_encode_submessage(ostream, FileInfo_fields, &fileinfo))
return false;
}
/* Because the main program uses pb_encode_delimited(), this callback will be
* called twice. Rewind the directory for the next call. */
rewinddir(dir);
}
return true;
}
/* Handle one arriving client connection.
* Clients are expected to send a ListFilesRequest, terminated by a '0'.
* Server will respond with a ListFilesResponse message.
*/
void handle_connection(int connfd)
{
DIR *directory = NULL;
/* Decode the message from the client and open the requested directory. */
{
ListFilesRequest request = {};
pb_istream_t input = pb_istream_from_socket(connfd);
if (!pb_decode_delimited(&input, ListFilesRequest_fields, &request))
{
printf("Decode failed: %s\n", PB_GET_ERROR(&input));
return;
}
directory = opendir(request.path);
printf("Listing directory: %s\n", request.path);
}
/* List the files in the directory and transmit the response to client */
{
ListFilesResponse response = {};
pb_ostream_t output = pb_ostream_from_socket(connfd);
if (directory == NULL)
{
perror("opendir");
/* Directory was not found, transmit error status */
response.has_path_error = true;
response.path_error = true;
}
else
{
/* Directory was found, transmit filenames */
response.has_path_error = false;
response.file = directory;
}
if (!pb_encode_delimited(&output, ListFilesResponse_fields, &response))
{
printf("Encoding failed: %s\n", PB_GET_ERROR(&output));
}
}
if (directory != NULL)
closedir(directory);
}
int main(int argc, char **argv)
{
int listenfd, connfd;
struct sockaddr_in servaddr;
int reuse = 1;
/* Listen on localhost:1234 for TCP connections */
listenfd = socket(AF_INET, SOCK_STREAM, 0);
setsockopt(listenfd, SOL_SOCKET, SO_REUSEADDR, &reuse, sizeof(reuse));
memset(&servaddr, 0, sizeof(servaddr));
servaddr.sin_family = AF_INET;
servaddr.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
servaddr.sin_port = htons(1234);
if (bind(listenfd, (struct sockaddr*)&servaddr, sizeof(servaddr)) != 0)
{
perror("bind");
return 1;
}
if (listen(listenfd, 5) != 0)
{
perror("listen");
return 1;
}
for(;;)
{
/* Wait for a client */
connfd = accept(listenfd, NULL, NULL);
if (connfd < 0)
{
perror("accept");
return 1;
}
printf("Got connection.\n");
handle_connection(connfd);
printf("Closing connection.\n");
close(connfd);
}
return 0;
}

5
lib/nanopb/examples/platformio/.gitignore vendored Normal file
View File

@@ -0,0 +1,5 @@
.pio/
.idea/
cmake-build-*/
/CMakeLists.txt
CMakeListsPrivate.txt

48
lib/nanopb/examples/platformio/platformio.ini Normal file
View File

@@ -0,0 +1,48 @@
;
; You can setup `custom_nanopb_protos` `nanopb_options` vars to generate code from proto files
;
; Generator will use next folders:
;
; `$BUILD_DIR/nanopb/generated-src` - `*.pb.h` and `*.pb.c` files
; `$BUILD_DIR/nanopb/md5` - MD5 files to track changes in source .proto/.options
;
; Compiled `.pb.o` files will be located under `$BUILD_DIR/nanopb/generated-build`
;
; Example:
[env:pio_with_options]
platform = native
lib_deps = Nanopb
src_filter =
+<pio_with_options.c>
; All path are relative to the `$PROJECT_DIR`
custom_nanopb_protos =
+<proto/pio_with_options.proto>
custom_nanopb_options =
--error-on-unmatched
[env:pio_without_options]
platform = native
lib_deps = Nanopb
src_filter =
+<pio_without_options.c>
; All path are relative to the `$PROJECT_DIR`
custom_nanopb_protos =
+<proto/pio_without_options.proto>
[env:pio_esp32_idf]
platform = espressif32
board = firebeetle32
framework = espidf
lib_deps = Nanopb
; Warning: the 'src_filter' option cannot be used with ESP-IDF. Select source files to build in the project CMakeLists.txt file.
; So, we specified source files in src/CMakeLists.txt
custom_nanopb_protos =
+<proto/pio_without_options.proto>

1
lib/nanopb/examples/platformio/proto/pio_with_options.options Normal file
View File

@@ -0,0 +1 @@
TestMessageWithOptions.str max_size:16

5
lib/nanopb/examples/platformio/proto/pio_with_options.proto Normal file
View File

@@ -0,0 +1,5 @@
syntax = "proto3";
message TestMessageWithOptions {
string str = 1;
}

5
lib/nanopb/examples/platformio/proto/pio_without_options.proto Normal file
View File

@@ -0,0 +1,5 @@
syntax = "proto3";
message TestMessageWithoutOptions {
int32 number = 1;
}

Some files were not shown because too many files have changed in this diff Show More