[作品提交] BLE超低功耗声纹门锁

xujinxi 2021-7-18 19:55 楼主

BLE超低功耗声纹门锁

作者：xujinxi

一、项目背景

OnSemi提供的RSL10蓝牙模组，应该为业界最小的蓝牙低功耗模组，并且该模组有业界最优的低功耗管理，且开发套件的sensor外设丰富，为各种奇思妙想的实现提供了非常理想的平台。

二、作品简介

RSL10蓝牙模组具有超低功耗以及方便使用的音频接口。搭载传感器以及相关的参考驱动代码，可以可靠、便捷的实现工程师的奇思妙想，深受工程师欢迎。EEWORLD+OnSemi开展的创意设计大赛，为广大工程师朋友提供施展才华的舞台。

本项目构思使用RSL10蓝牙模组搭载的DMIC和IMU单元。用于智能门锁户外敲门的识别和语音拾取，并发送至室内接收端。室内的RSL10蓝牙模组则接收户外传来的语音数据，通过串口传递至PC，PC使用该语音数据访问语音平台，识别语音指令。如果是正确的“开门”指令，则”开门”信号通过串口经室内RSL10蓝牙模组，再通过BLE传输至户外RSL10蓝牙模组，进而发出门锁开启信号。完成整个低功耗语音门锁的实现

三、系统框图和各部分功能说明

整个体统主要由RSL10、IMU、DMIC等传感器接口组成，如下为系统电子系统框图：USB端口连接Jlink，Jlink通过USB CDC建立与PC的通讯路径，RSL10户外语音采集板将采集的语音数据通过蓝牙传至室内RSL10模组，再通过Jlink的串口上传至PC Python代码，用于数据分析和处理。PC将数据推送至百度语音识别引擎，识别结果返回给PC。最后下发至室内和户外RSL10模组做相应操作。

四、作品源码和案例中处理的传感器数据

central_client_uart_on_evb-户内机.7z (106.3 KB)
(下载次数: 6, 2021-7-28 09:29 上传)

peripheral_server_uart_on_sensor-户外机.7z (491.09 KB)
(下载次数: 6, 2021-7-28 09:29 上传)

RSL10 BLE超低功耗声纹门锁.py (3.95 KB)
(下载次数: 5, 2021-7-28 09:30 上传)

/* ----------------------------------------------------------------------------
* Copyright (c) 2015-2017 Semiconductor Components Industries, LLC (d/b/a
* ON Semiconductor), All Rights Reserved
*
* Copyright (C) RivieraWaves 2009-2016
*
* This module is derived in part from example code provided by RivieraWaves
* and as such the underlying code is the property of RivieraWaves [a member
* of the CEVA, Inc. group of companies], together with additional code which
* is the property of ON Semiconductor. The code (in whole or any part) may not
* be redistributed in any form without prior written permission from
* ON Semiconductor.
*
* The terms of use and warranty for this code are covered by contractual
* agreements between ON Semiconductor and the licensee.
*
* This is Reusable Code.
*
* ----------------------------------------------------------------------------
* app.c
* - Main application file
* ----------------------------------------------------------------------------
* $Revision: 1.25 $
* $Date: 2017/12/05 16:02:54 $
* ------------------------------------------------------------------------- */

//20210626 add dmic support by hhz
#include "app.h"
#include <printf.h>

#define DMIC_DATA_WIDTH_16 //16bit DMIC data-- DMIC_DATA_WIDTH_8 8bit DMIC data

#ifdef DMIC_DATA_WIDTH_8       //FS = 16000hz,data width = int8_t,2 second record data =16000*1 =16000 samples
   #define DMIC_BUF_SIZE 16000       //100 sample, equal 200 bytes data(int16_t)
#else
   #define DMIC_BUF_SIZE 16000       //FS = 16000hz,data width = int16_t,2 second record data =16000*2 =32000 s
#endif

#define BLE_TX_PACKET_SIZE 100 //200

uint32_t output_flag = 0;
uint32_t data_ready_flag = 0; //init not data ready until interruption receive DMIC_BUF_SIZE data from DMIC

int16_t dmic_value_scope_int16 = 0;

#ifdef DMIC_DATA_WIDTH_8
int8_t dmic_value_buf[DMIC_BUF_SIZE] = {0}; //d0_msb,d0_lsb,d1_msb,d1_lsb...
#else
int16_t dmic_value_buf[DMIC_BUF_SIZE] = {0}; //d0_msb,d0_lsb,d1_msb,d1_lsb...
#endif

//dmic_value_buf[0] = lsb, dmic_value_buf[1] = msb
uint8_t* dmic_buf_pointer_uint8 = (uint8_t *)dmic_value_buf;

uint32_t j = 0,k = 0;

uint32_t dmic_packet_num = 0;
uint32_t tx_cnt = 0;
uint32_t record_sample_num = 0;
//+ lsb_num,~,~,msb_num packet num uint32 lsb fist
uint8_t* pointer_record_sample_num_uint8 = (uint8_t *)(&record_sample_num);

uint8_t syn_head_array[4] = {0x55,0x55,0x55,0x55}; //+ lsb_num,~,~,msb_num packet num uint32 lsb fist
uint8_t syn_end_array[4] = {0xaa,0xaa,0xaa,0xaa}; //+ lsb_num,~,~,msb_num packet num uint32 lsb fist
uint8_t head_sended_flag = 0;
uint8_t end_sended_flag = 0;

static uint8_t asr_flag = 0;

void DMIC_OUT_OD_IN_IRQHandler(void)
{
static int32_t i = 0;

//dmic_value_scope_int32 = (int32_t)AUDIO->DMIC0_DATA;
//dmic_value_scope_int16 = (int16_t)((dmic_value_scope_int32 >> 1) & 0xffff);
//dmic_value_scope_int16 = (int16_t)((dmic_value_scope_int32) & 0xffff);
//dmic_value_scope_int16 = (int16_t)(dmic_value_scope_int32);
//PRINTF("%d = %08x = %d\n",dmic_value_scope_int32,dmic_value_scope_int32,dmic_value_scope_int16);

//dmic_value_scope_int16 = (int16_t)AUDIO->DMIC0_DATA;
dmic_value_scope_int16 = (int16_t)((uint16_t)(AUDIO->DMIC_DATA & 0x0000ffff));
Sys_GPIO_Toggle(3); //test frame frequency

//dmic_value_scope_int16 = dmic_packet_num;
//PRINTF("%d = %08x\n",dmic_value_scope_int16,dmic_value_scope_int16);

if(dmic_packet_num < DMIC_BUF_SIZE)
{
       #ifdef DMIC_DATA_WIDTH_8       //FS = 16000hz,data width = int8_t,2 second record data =16000*1 =16000 samples
       dmic_value_buf[dmic_packet_num] = (int8_t)dmic_value_scope_int16;
       #else
       dmic_value_buf[dmic_packet_num] = dmic_value_scope_int16;
       //dmic_value_buf[dmic_packet_num] = dmic_packet_num;
       #endif

   dmic_packet_num++;
}
else
{
   data_ready_flag = 1;
   //asr_flag = 2;
   //NVIC_DisableIRQ(DMIC_OUT_OD_IN_IRQn);       //close DMIC interrupt
}

}

void DIO0_IRQHandler(void)
{
//static uint8_t ignore_next_dio_int = 1;
PRINTF("DIO0 INTERRUPT\n");
asr_flag ++;

if(asr_flag == 2)
NVIC_EnableIRQ(DMIC_OUT_OD_IN_IRQn);
}

int main(void)
{
uint32_t length;
uint8_t temp[BUFFER_SIZE]; //for uart

static uint8_t time_cnt = 0;

/* Initialize the system */
App_Initialize();

//setting dio3 as dmic interrupt toggle output for sample rate conform
Sys_DIO_Config(3, DIO_MODE_GPIO_OUT_0); //CLOCK OUTPUT
//Sys_DIO_Config(3, DIO_MODE_AUDIOCLK);

//Sys_DIO_Config(LED_DIO_GREEN, DIO_MODE_GPIO_OUT_0); //GREEN LED
Sys_GPIO_Toggle(LED_DIO_GREEN);
Sys_GPIO_Set_High(LED_DIO_GREEN);

NVIC_EnableIRQ(DIO0_IRQn);

PRINTF("HARDWARE INIT OK !\n"); //hhz
PRINTF("into main loop !\n"); //hhz

/* Main application loop:
* - Run the kernel scheduler
* - Send notifications for the battery voltage and RSSI values
* - Refresh the watchdog and wait for an interrupt before continuing */
while (1)
{
Kernel_Schedule();

/*if(asr_flag == 2)
{
   NVIC_EnableIRQ(DMIC_OUT_OD_IN_IRQn);
   //PRINTF("NVIC_EnableIRQ\n");
}
else if(asr_flag == 0)
{
   NVIC_DisableIRQ(DMIC_OUT_OD_IN_IRQn);
   //PRINTF("NVIC_DisableIRQ\n");
}*/

if (unhandled_packets != NULL)
{
if (UART_FillTXBuffer(unhandled_packets->length,
unhandled_packets->data) !=
UART_ERRNO_OVERFLOW)
{
unhandled_packets = removeNode(unhandled_packets);
}
}

if (ble_env.state == APPM_CONNECTED)
{
if (app_env.send_batt_ntf && bass_support_env.enable)
{
app_env.send_batt_ntf = 0;
Batt_LevelUpdateSend(0, app_env.batt_lvl, 0);
}

if (cs_env.sentSuccess)
{
// Copy data from the UART RX buffer to the TX buffer
length = UART_EmptyRXBuffer(temp);
if (length > 0)
{
// Split buffer into two packets when it's greater than
// packet size
if (length > PACKET_SIZE)
{
CustomService_SendNotification(ble_env.conidx,
CS_IDX_TX_VALUE_VAL,
temp,
PACKET_SIZE);
CustomService_SendNotification(ble_env.conidx,
CS_IDX_TX_VALUE_VAL,
&temp[PACKET_SIZE],
length - PACKET_SIZE);
}
else
{
CustomService_SendNotification(ble_env.conidx,
CS_IDX_TX_VALUE_VAL,
temp,
length);
}
}
}

//test dmic first 0~1000 sample data
/*
if((data_ready_flag == 1) & (tx_cnt < DMIC_BUF_SIZE))
{
PRINTF("%d\n",dmic_value_buf[tx_cnt]);
tx_cnt++;
}
*/

//if (cs_env.sentSuccess)
if (cs_env.sentSuccess & (data_ready_flag == 1))
{
head_sended_flag = 1; //without head test
end_sended_flag = 1; //without end test

   if(head_sended_flag == 0)
   {
       PRINTF("TX:syn_head_array !\n");   //hhz
       CustomService_SendNotification(ble_env.conidx,
   CS_IDX_TX_VALUE_VAL,
                                               syn_head_array,
                                               4);
   //CustomService_SendNotification(ble_env.conidx,
   //     CS_IDX_TX_VALUE_VAL,
                   //                           pointer_record_sample_num_uint8,
                   //                           4);
       head_sended_flag = 1;
   }

#ifdef DMIC_DATA_WIDTH_8
for(tx_cnt=0;tx_cnt < DMIC_BUF_SIZE/BLE_TX_PACKET_SIZE;tx_cnt++)
   {
       PRINTF("TX:%d ' DMIC packet !\n",tx_cnt);
       CustomService_SendNotification(ble_env.conidx,
   CS_IDX_TX_VALUE_VAL,
                                               dmic_buf_pointer_uint8,
                                               BLE_TX_PACKET_SIZE);
       dmic_buf_pointer_uint8 += BLE_TX_PACKET_SIZE;

       PRINTF("TX:tx dmic watchdog refresh !\n");
       Sys_Watchdog_Refresh();
   }
#else
if((data_ready_flag == 1) & (tx_cnt < DMIC_BUF_SIZE*2/BLE_TX_PACKET_SIZE))
{
   //PRINTF("%d\n",dmic_value_buf[tx_cnt]);
   PRINTF("TX:%d ' DMIC packet !\n",tx_cnt);

   //CustomService_SendNotification(ble_env.conidx,
   //        CS_IDX_TX_VALUE_VAL,
   //                               syn_head_array,
   //                               2);

   CustomService_SendNotification(ble_env.conidx,
       CS_IDX_TX_VALUE_VAL,
                                           dmic_buf_pointer_uint8,
                                           BLE_TX_PACKET_SIZE);

dmic_buf_pointer_uint8 += BLE_TX_PACKET_SIZE;

   //CustomService_SendNotification(ble_env.conidx,
   //     CS_IDX_TX_VALUE_VAL,
   //                 syn_end_array,
   //                 2);

tx_cnt++;
}
else if((data_ready_flag == 1) & (tx_cnt >= DMIC_BUF_SIZE*2/BLE_TX_PACKET_SIZE))
{
data_ready_flag = 0;

   //without head test
   //head_sended_flag = 0;
   //end_sended_flag = 0;
   dmic_buf_pointer_uint8 = (uint8_t *)dmic_value_buf;
   dmic_packet_num = 0;
   tx_cnt = 0;
   asr_flag = 0;
   NVIC_DisableIRQ(DMIC_OUT_OD_IN_IRQn);
   PRINTF("close od interrupt,init all flag\n");
}
   /*for(tx_cnt=0;tx_cnt < DMIC_BUF_SIZE*2/BLE_TX_PACKET_SIZE;tx_cnt++)
   {
       PRINTF("TX:%d ' DMIC packet !\n",tx_cnt);
       CustomService_SendNotification(ble_env.conidx,
   CS_IDX_TX_VALUE_VAL,
                                               dmic_buf_pointer_uint8,
                                               BLE_TX_PACKET_SIZE);
       dmic_buf_pointer_uint8 += BLE_TX_PACKET_SIZE;

       PRINTF("TX:tx dmic watchdog refresh !\n");
       Sys_Watchdog_Refresh();
   }*/

#endif
if((end_sended_flag == 0) & (data_ready_flag == 0))   //all dmic data have been sended over BLE
{
   PRINTF("TX:syn_end_array !\n");   //hhz
   CustomService_SendNotification(ble_env.conidx,
   CS_IDX_TX_VALUE_VAL,
                                               syn_end_array,
                                               4);
   //end_sended_flag = 1;

head_sended_flag = 0;
end_sended_flag = 0;

   dmic_buf_pointer_uint8 = (uint8_t *)dmic_value_buf;
   dmic_packet_num = 0;
   tx_cnt = 0;
   asr_flag = 0;
   PRINTF("close od interrupt,init all flag\n");
}

//PRINTF("TX dmic_value_buf\n");

//for(tx_cnt=0;tx_cnt < DMIC_BUF_SIZE;tx_cnt++)
//{
// PRINTF("%d\n",dmic_value_buf[tx_cnt]);
// Sys_Watchdog_Refresh();
//}

data_ready_flag = 0;
}

//Sys_Watchdog_Refresh();

} //connected

Sys_GPIO_Toggle(3);

/* Refresh the watchdog timer */
Sys_Watchdog_Refresh();

/* Wait for an event before executing the scheduler again */
SYS_WAIT_FOR_EVENT;

} //while(1)
} //main

五、视频演示

1、室内机RSL10模组上电；

2、户外RSL10 sensor板上电；

3、启动PC处理程序，连接室内RSL10模组；

4、通过按键启动户外RSL10 sensor模组录音，录音数据通过BLE传输至室内RSL10模组，RSL10模组转接至PC，PC转百度语音识别引擎识别，识别结果返回PC，并做识别结果演示。

演示视频链接如下：

B站：https://www.bilibili.com/video/BV1Ff4y1j7KW?share_source=copy_web

优酷：https://v.youku.com/v_show/id_XNTE4MzEzNDMwNA==.html

六、项目总结

RSL10开发板撘载的多种sensor：例如IMU，DMIC，温度，湿度等传感器，在简单或者复杂环境下，可以同时采集多种数据用于数据分析，以找出环境与传感器的数据关系，从而方便电子设备辨别环境或者动作，实现电子系统对环境和动作的感知。相关的代码例程和sensor硬件DEMO，方便工程师将传感器集成进原来系统内，进而减少时间浪费，快速评估传感器，赞👍！

本项目只完成了户外RSL10模组DMIC数据采集，DMIC数据BLE传输至室内RSL10模组，RSL10通过串口传输音频数据到PC，并通过百度语音识别引擎完成语音识别，并输出识别结果。剩余功能未完成。另DMIC数据以及传输可靠性还需优化，成功率较低。

OnSemi作品提交-BLE超低功耗声纹门锁 .doc (2021-7-19 23:11 上传)