E.R.V.S - The Every Room Voice Sensor Board

Voice Assistant Contest
1

My Year of the Voice follow-up project is a board that I can use around the house to control HA from anywhere and monitor what’s going on. So I started researching the best MCU for this and ended up with the ESP32-S3-WROOM-1-N16R8 which has a lot of options to play with. (16kb Flash and 8mb PSRAM, 1u can be used for external antenna).
Main PCB front and back, dimension 80x80mm:

PCB-Front
PCB-Front801×800 27 KB
PCB-Back
PCB-Back801×800 15.3 KB

The voice assist part is handled by 4 onboard PDM MEMS microphones, which can be extended to 8 using breakout boards. So there are 4 microphones facing the front and 4 microphones facing the sides, which gives it a 180° spherical voice detection.
MIC-PCB front and back, dimension 13x6mm:

pdm-mic-f
pdm-mic-f789×367 5.17 KB
pdm-mic-b
pdm-mic-b789×367 4.87 KB

But to make it more than just a voice assistant, there are 4(-5) extra sensors:
LD2410(b/c) or LD2450 for presence and position detection to automate things when you move or when you are in the room.
BH1750 to measure light intensity and control lighting when it’s too dark.
BME680 to get temperature, humidity and air pressure: so I can control the heating, air conditioning and whatever I want. With the VOC gas detector I also have a smoke and fire alarm, which I hope I never need!
Lastly, I have an SCP40 that measures VOC and NOx to give me an indication of air quality.

For feedback on actions and status in the room/area of the device, there are 6 addressable LEDs to show what’s going on. 1 LED is for planned for air quality (green to red) and 5 LEDs give feedback for voice assistance (available, listing, WW detected, working, success, failure). Still working on it and thinking about using LED 1 and 6 for indication and 4 LEDs in the middle for VA.
A buzzer is there to warn if something is wrong and to annoy in case of fire.
Last but not least, 2 headers are broken out to connect a speaker via i2s amplifier and i2c sensors if I want to use it later. 5 extra unused pins on the back of the board give me even more room to play with!

All the design, research and component testing took a lot of time, so I just finished the prototype PCB and ordered it fully assembled. I hope to get it before the content deadline and show some working stuff on it. Until then, I’m working on the yaml files for it, based on the work of the community and the HA/ESPhome developers. Eventually I would like to make the whole design, PCB, Gerber files and source code public and open source, so that anyone can order the HA-E.R.V.S board and use it as they wish.
I hope you will accept my participation in the contest, even if I can’t show the working board and make a video yet, but I will try my best to submit it as soon as possible.

2

Interesting. From experience, know that the LD24XX themselves generate heat, so you’ll likely have a hard time to have your BMEx80 returning accurate values.

2 Likes
3

I’ve tested with a case where the Temp is at the bottom and LD24** at the top with passive ventilation from bottom to top and the difference between bottom of room and ceiling was greater then with/-out LD****. :sweat_smile: of course, you can’t use a totally closed case. Then also smoke detection doesn’t work. So the plan in the beginning is to use the hot air from LD**** to suck in more air and have better ventilation and “fresh” air at the sensors at the bottom center.
But with a temp offset in code, this can also be fixed.

3 Likes
4

@MoritzS I really like your project and will follow closely as you are building the type of device I’ve been looking for quite a while. As for the temperature offset in code, that would require the heat from the LD2410 to affect the BME680 always in the same way. While I don’t have a way to perform professional measurements, I got the impression that it’s effect varies with the environmental temperature so a fixed offset may not be ideal. What do you think?

2 Likes
5

There are several issues I already know and you didn’t think about it yet. :sweat_smile:

  • Status LED which indicates air quality and light sensor.:person_facepalming:
  • Heat and temperature
  • Radar waves from 2 devices
  • Air quality pending on humidity

If you look closer at the area above the LED (where LED1-6 is written), there is a 80x3mm area with no parts at all. This is supposed to be covered by a black plastic piece with 2 wholes at the side. So you have an air flow from center bottom, where sensors are, to the side and top where LD2410/50 and BH1750 are.
My hope is to isolate the 2 groups that much, it can’t effect each other. As said before, I’ll have ventilation holes in the case at some spots.
I have an IR thermostat to measure Temp while with/-out mmWave and if there is a continuous higher temp of 1° this can be offset by code. It’s not that the LD24** produce lot of heat one time while on, and no heat other time. So the offset should be always the same (with air coming from bottom first).

6

This is an interesting concept. I hope it works out as expected.

7

Besides the microphone, everything works out great at the moment. I already set it up on breadboard while PCB are in production and code is almost finished. Specially the AQS LED and presence detection with LED indicator is nice. Still working on optimisation and try to work out which entity’s I should disable, hide and group. I have over 140 values so far.
VA is as bad as with every single INMP441 microphone I have to admin. Hopefully there will be a big improvement by the mic array.:sweat_smile:
PCB are in production and will be here end of month I hope.

1000090692
10000906921920×1440 225 KB

1000090691
10000906911920×1440 148 KB

1 Like
8

I am wondering if there is already a way to integrate multiple microphones as an array, e.g. the mentioned INMP441, using ESPHome? Or what kind of microphones will be used on the final board?

9

No, esphome and VA can only have one mic for left or right channel. That’s why I’m using up to 8 pdm microphones with es7210. The 4 onboard are left channel and the 4 additional are right channel.

The mics I’m using are MP34DT05TR-A. With a sensitivity of -26dB and 64dB SNR they sounded promising for the price of 1$.

10

So, according to jlcpcb, I will actually get the PCB prototypes before the contest deadline. They should be finished tomorrow and shipped early this week.
I kinda finished my codebase and besides pin mapping it works like a charm now. I have a nice display of air quality, movement, still presence and target detection. I already dropped the sgp40 sensor and thought “what would I change?”.
So while playing around with code and layout, I actually tried to add a camera interface and split i2s between mic in and speaker out :smiley: So now, if I didn’t screw up the ES7210 circuit and mic array, I’ll extend my boards with OV5640 and be able to send me a picture of the environment when I have bad air quality or unexpected motion detection.

ERVS-Beta1
ERVS-Beta1783×782 27.8 KB

Fun fact: I used every pin avaible on ESP32-S3-Wroom-1…

ESP32-Pinmapping
ESP32-Pinmapping728×789 11.5 KB

Also I’ve moved the LEDs to the very bottom, moved around the microphones, USB connector and light sensor. The code is now on Github and when the PCB are here, I can test the final version and Pin-Mapping. Still need to add more comments and cleanup to fix copy and paste errors.

5 Likes
11

I was pondering over a project like this, and have been wondering how well XVF-3000 would work for the mic array.
I’m hoping that the eventual hardware the community comes up with will be able to function as a media renderer too, I really want to get rid of those Alexa devices :wink:

12

The XVF-3000 is used in respeaker AMD other boards, but I don’t think it can be used with esp32 or similar.
I’m playing around with Voice now for half a year to get rid of Alexa and I don’t think it’s possible by the current implementation in HA. Specially not if you’re using esp32 as satellites…
https://heywillow.io/ is a better approach, but it’s harder to install and setting up the WIS server to get it away from someones cloud. Also wake words are an issue and I don’t like much, willow requires the S3-BOX-3 with touch and no other hardware and sensors… It would be great to have esphome style installation for Willow and customization, but who knows what comes in the future…

By the way, I’ve played around with my PCBs today and lot of failures. Everything is working as expected, besides the es7210. Need to figure this issues tomorrow and get voice up and running. At the moment, the es7210 gets recognized by VA, but the is in a mute state and no voice gets picked up. :sob:

13

Ah, yes after looking over the data sheet again it does look like it’s a USB board. Possibly the XVF3510 could be the version that could work, having I2S and I2C lines for communication.

Unfortunately I’m also thinking the same thing with the ESP32, it’s a wonderful chip but it’s probably not best suited to handle everything we want/need from a satellite.

As for the PCB’s awesome to hear that they work bar a few sensors. For now possibly try to not bite off more than you can chew and focus on improving what works. At the moment I think the ideal solution would be something that can replicate/replace an Echo Dot V3, i.e far field audio and a decent playback speaker, with the addition of a couple of simple sensors (Temp, humidity, lux)

14

Well, I was reading hunderts of pages last night and this morning to figure out where I f***ed it… It doesn’t look like it’s the hardware, but the ES7210 components and ESP-ADF handling of pins and busses. For example

and

shows I can’t use “my” I2C Pins, because they are diffrent from adf and so

external_components:
  - source: github://rpatel3001/esphome@es7210
    components: [ es7210 ]
  - source: github://pr#5230
    components:
      - esp_adf

does not work. I’m trying today to add a I2S Mic on external pin header and change my pin config completly to match S3-Box-3 for next PT version. This way I could use

without Display part, add camera part from esp32-s3-korvo-2 or S3-EYE… Monday also my pin headers arrive, so I can mount the LD2410 and LD2450 (forgot to order them and don’t want to solder them on PT-PCB). Still longer way to go, but I’m close.

15

So after the weekend and testing, adjusting and playing with the prototype, here are few videos. Sry about my bad English, it was a long day and short night. :sweat_smile:
Voice control:

Board and sensors explained:

The Air quality sensor and adjustments:

And last but not least: overview of esphome integration:

Still working on get 4-8 microphones working, but I’m happy it works for now within 3m in a silent room to control by voice and monitor movement, presence and air quality, lux and temperature/humidity. Basically everything I need in every room in future to get rid of all sensors to switch on/off light and heating.

4 Likes
16

nice board. let me know if you sell it one day :slight_smile:

1 Like
17

looks great - also interessted to buy it :slight_smile:

18

I will not sell it. I’m still struggling with esp_adf and need some more time and money for few more prototype PCB with es7210 and speakers, but when I’m done, it will be open source.

3 Likes
19

Love to see more people doing open source. When you release I’ll build one and see if there is anything I can add.