JTAG debugging ESP32 - huzzah32

I am currently working on a project using an ESP32. I really the way Espressif has set up the development tools, I’ll probably do a post about my thoughts about it.

The serial port and logging is setup out of the box, so I’ve been using printfs to debug the code. There are a couple of problems with this approach, but one big advantage, it is easy and just works. The downsides are that for every time you want to look at a variable, you add a printf, you build the program and need to upload it to the board, and as it is uploaded through the bootloader operating at 115200baud it takes some time.

While I was programming it always seemed better to focus on the code and not to spend time on setting up the debugging. Now I have some code typed up, it would be a great time to setup JTAG debugging.

When you have JTAG debugging setup you have a lot more than using printfs. You can watch local variables while single stepping through lines of code, you can stop the execution when a variable or a memory location equals some value. It is a very powerful thing. So how do we do it and what are the pitfalls?

The hardware I am using to do this is:

Here you can find the schematic for the feather esp32 board and for the ftdi breakout board.

Connecting everything up

You need to connect these pins

ESP32 / FT2232

  1. MTDI - GPIO12 / ADBUS1 - TDI
  2. MTCK - GPIO13 / ADBUS0 - TCK
  3. MTMS - GPIO14 / ADBUS3 - TMS
  4. MTDO - GPIO15 / ADBUS2 - TDO
  5. GND / GND

Let me get one thing out of the way before I start talking about setting up the software. LadyAda feather board has two leds. The orange one is connected to the charging status pin of the charger. The other one, the red one is connected to the esp32 pin 13. So if you were using the led as a heartbeat signal, or for debugging, or whatever, do not do it any more, because that pin is needed for the jtag connection. This spent about an hour of my time :-)

Setting up the software

First of all I will presume that you have esp-idf setup, and that you have basic experience of using it, and that this is not your first time setting up a jtag gdb debugging ie. I will not explain the basic things

It seems that the espressif guys modified a bit the ocd/gdb protocol and it is still not upstreamed to the openocd project, so we will need to use espressifs repo. You can get compiled releases here. Just download the latest version that is appropriate for the development machine that you are using and extract it somewhere.

I did a lot of modifications to the scripts used to connect to board with different configurations trying to figure out what is the correct configuration. As I said before in the end I figured out that the problem was that my program running on the esp was using one of the jtag pins. So in the end there are no modifications that need to be done.

Running openocd

If everything is connected correctly we should be able to run openocd like this from the bin folder:

./openocd -f ../share/openocd/scripts/interface/ftdi/esp32_devkitj_v1.cfg -f ../share/openocd/scripts/board/esp-wroom-32.cfg

You should see an output similar to this:

Open On-Chip Debugger  v0.10.0-esp32-20190708 (2019-07-08-11:03)
Licensed under GNU GPL v2
For bug reports, read
	http://openocd.org/doc/doxygen/bugs.html
none separate
adapter speed: 1000 kHz
Info : Configured 2 cores
esp32 interrupt mask on
Info : Listening on port 6666 for tcl connections
Info : Listening on port 4444 for telnet connections
Info : clock speed 1000 kHz
Info : JTAG tap: esp32.cpu0 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
Info : JTAG tap: esp32.cpu1 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
Info : Target halted. PRO_CPU: PC=0x40007BED (active)    APP_CPU: PC=0x00000000 
Info : Listening on port 3333 for gdb connections
Info : esp32: Debug controller 0 was reset (pwrstat=0x5F, after clear 0x0F).
Info : esp32: Core 0 was reset (pwrstat=0x5F, after clear 0x0F).
Info : esp32: Debug controller 1 was reset (pwrstat=0x5F, after clear 0x0F).
Info : esp32: Core 1 was reset (pwrstat=0x5F, after clear 0x0F).
Info : Detected debug stubs @ 3ffb37c4 on core0 of target 'esp32'
Info : accepting 'gdb' connection on tcp/3333
Error: No symbols for FreeRTOS
Info : Target halted. PRO_CPU: PC=0x4015FE96 (active)    APP_CPU: PC=0x4015FE96 
Info : Target halted. PRO_CPU: PC=0x400916EE (active)    APP_CPU: PC=0x4015FE96 
Info : Flash mapping 0: 0x10020 -> 0x3f400020, 106 KB
Info : Flash mapping 1: 0x30018 -> 0x400d0018, 581 KB
Info : Target halted. PRO_CPU: PC=0x400916EE (active)    APP_CPU: PC=0x4015FE96 
Info : Auto-detected flash size 4096 KB
Info : Using flash size 4096 KB
Info : Target halted. PRO_CPU: PC=0x400916EE (active)    APP_CPU: PC=0x4015FE96 
Info : Flash mapping 0: 0x10020 -> 0x3f400020, 106 KB
Info : Flash mapping 1: 0x30018 -> 0x400d0018, 581 KB
Info : Using flash size 584 KB
Info : Target halted. PRO_CPU: PC=0x400916EE (active)    APP_CPU: PC=0x4015FE96 
Info : Flash mapping 0: 0x10020 -> 0x3f400020, 106 KB
Info : Flash mapping 1: 0x30018 -> 0x400d0018, 581 KB
Info : Using flash size 108 KB
Warn : negative reply, retrying
Info : JTAG tap: esp32.cpu0 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
Info : JTAG tap: esp32.cpu1 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
Info : esp32: Debug controller 0 was reset (pwrstat=0x5F, after clear 0x0F).
Info : esp32: Core 0 was reset (pwrstat=0x5F, after clear 0x0F).
Info : esp32: Debug controller 1 was reset (pwrstat=0x5F, after clear 0x5F).
Info : esp32: Core 1 was reset (pwrstat=0x5F, after clear 0x5F).
Info : Target halted. PRO_CPU: PC=0x5000004B (active)    APP_CPU: PC=0x00000000 
Info : esp32: Core 0 was reset (pwrstat=0x1F, after clear 0x0F).
Info : Target halted. PRO_CPU: PC=0x40000400 (active)    APP_CPU: PC=0x40000400 
Info : Target halted. PRO_CPU: PC=0x400D621C (active)    APP_CPU: PC=0x4015FE96 
Info : Detected debug stubs @ 3ffb37c4 on core0 of target 'esp32'

In the scripts/board/esp-wroom-32.cfg there is a line starting with adapter_khz which sets the JTAG speed. It is, a bit optimistically, set to 20MHz, so if this is not really working that great, try lowering the speed, especially if you are connecting the boards with longer wires.

Running xtensa-esp32-elf-gdb

In the executable files that come in the esp-idf there is xtensa-esp32-elf-gdb that we will use to connect to the openocd.

There is a new file, lets call it gdbinit, that you need to make with the lines below. We will use that file when starting gdb.

target remote :3333 
set remote hardware-watchpoint-limit 2 
mon reset halt 
flushregs 
thb app_main 
c

After we have made the file, and started the openocd, we will start the gdb and connect to the openocd.

xtensa-esp32-elf-gdb -x gdbinit ~/path/to/elf/file.elf

If everything is running ok, the output should look like this:

GNU gdb (crosstool-NG esp32-2019r1) 8.1.0.20180627-git
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "--host=x86_64-build_pc-linux-gnu --target=xtensa-esp32-elf".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/lovro/code/cunjovatikljunas-esp32/ckfw/build/ckfw.elf...done.
0x4015fe96 in esp_pm_impl_waiti ()
    at /home/lovro/code/cunjovatikljunas-esp32/esp-idf/components/esp32/pm_esp32.c:493
493	    asm("waiti 0");
JTAG tap: esp32.cpu0 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
JTAG tap: esp32.cpu1 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
esp32: Debug controller 0 was reset (pwrstat=0x5F, after clear 0x0F).
esp32: Core 0 was reset (pwrstat=0x5F, after clear 0x0F).
esp32: Debug controller 1 was reset (pwrstat=0x5F, after clear 0x5F).
esp32: Core 1 was reset (pwrstat=0x5F, after clear 0x5F).
Target halted. PRO_CPU: PC=0x5000004B (active)    APP_CPU: PC=0x00000000 
esp32: Core 0 was reset (pwrstat=0x1F, after clear 0x0F).
Target halted. PRO_CPU: PC=0x40000400 (active)    APP_CPU: PC=0x40000400 
Hardware assisted breakpoint 1 at 0x400d621c: file ../main/ckfw_main.c, line 173.
Target halted. PRO_CPU: PC=0x400D621C (active)    APP_CPU: PC=0x4015FE96 
[New Thread 1073462640]
[New Thread 1073460740]
[New Thread 1073465296]
[New Thread 1073453032]
[New Thread 1073412864]
[New Thread 1073413736]
[New Thread 1073454164]
[Switching to Thread 1073458708]

Thread 1 hit Temporary breakpoint 1, app_main () at ../main/ckfw_main.c:173

You can now use gdb!

Integrating debugging into VScode

Using gdb by hand is all good, and doing some more complicated things is easier to do by hand directly in gdb. But the usual stuff you will do is nicer and quicker to do visually in the gui.

TBD!!!