Embedded systems debugging 25 February 2015 Lecture

Agenda 1.Makefiles 1.What is a makefile? 2.Syntax basics 3.Variables, automatic variables, dependency chains, phony targets 2.GDB 1.What is GDB? 2.Basic commands 3.Coredump analysis 4.GDBServer 5.GDB in Eclipse 3.Cross compilation

Makefiles

What is a makefile Basic compilation: g++ -c app.cpp -o app.o What if we have many files? g++ -c file1.cpp file2.cpp file3.cpp -o app.o Cons: Compiling everything, everytime; Doing everything „by hand”; No fast cleaning procedure; Makefiles

What is a makefile Makefile – text file written in a certain prescribed syntax, helps in building software, organizing code, compilation, linking and clearing the workspace. It can be understood as a recipe to acquiring predefined targets. Standard targets often contain: Build, rebuild, clean, configure, install; Makefiles

Syntax basics Makefiles

Variables CC=g++ main: main.o $(CC) main.o -o main main.o: main.cpp $(CC) -c main.cpp -o main.o Makefiles

Automatic variables Makefiles Available automatic variables: - current target, everything on the left of the colon; $^ - all of the dependencies, everything on the right of the colon; $< - first file from the list of dependencies (on the right of the colon); $? - all of the dependencies which are out of date; $* - a call to the matched pattern, selected using %. Example: CC=g++ CFLAGS=-c -Wall main: main.o file1.o $(CC) $^ -o %.o: %.cpp $(CC) $(CFLAGS) $< -o

Dependency chains Makefiles CC=g++ CFLAGS=-c -Wall all: hello hello: main.o factorial.o hello.o $(CC) main.o factorial.o hello.o -o hello main.o: main.cpp $(CC) $(CFLAGS) main.cpp factorial.o: factorial.cpp $(CC) $(CFLAGS) factorial.cpp hello.o: hello.cpp $(CC) $(CFLAGS) hello.cpp clean: rm -f *o hello

Phony targets Makefiles CC=g++ CFLAGS=-c -Wall all: hello hello: main.o factorial.o hello.o $(CC) main.o factorial.o hello.o -o hello main.o: main.cpp $(CC) $(CFLAGS) main.cpp factorial.o: factorial.cpp $(CC) $(CFLAGS) factorial.cpp hello.o: hello.cpp $(CC) $(CFLAGS) hello.cpp.PHONY:clean clean: rm -f *o hello

GDB

What is GDB? GDB GNU Debugger – standard debugger for GNU operating systems, works with Ada, C, C++, Fortran, Java, etc... Enables running application in a controlled environment, variable viewing/substitution, backtracing, core dump analysis, and more.

Debug symbols GDB gcc -g produces debugging information, so that GDB can work with it. You can also strip this information with gcc –s, or strip command, in order to reduce binary size. This, along with optimization flags, are often the two main differences between release and debug software versions, as seen from a toolchain level.

Optimization GDB Debugging a program built with –g and –O may produce unexpected results. Shortcuts taken by optimized code may contain: Some variables not existing at all; Flow of control moving to unexpected places; Some statements computation ommited, due to their constant state or values already at hand; Statements executing in other places, as they may have been moved out of loops;

Basic commands GDB gdb./app – run application under gdb control; Example application: #include int main(int argc, char* argv[]) { int test=5; void* buf = malloc(16); free(buf); printf("test %d", test); return 0; }

GDB Example application: #include int main(int argc, char* argv[]) { int test=5; void* buf = malloc(16); free(buf); printf("test %d", test); return 0; } programming]$ gdb./main GNU gdb (GDB) Fedora ( fc14) Copyright (C) 2010 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "i686-redhat-linux-gnu". For bug reporting instructions, please see:... Reading symbols from /home/build/programming/main...done. (gdb) run Starting program: /home/build/programming/main *** glibc detected *** /home/build/programming/main: double free or corruption (fasttop): 0x0804a008 *** ======= Backtrace: ========= /lib/libc.so.6[0xaf7fb6] /home/build/programming/main[0x804845d] /lib/libc.so.6(__libc_start_main+0xe6)[0xa9fe36] /home/build/programming/main[0x ] ======= Memory map: ======== r-xp :00 0 [vdso] 00a a85000 r-xp fd:00 75 /lib/ld-2.13.so 00a a86000 r--p 0001c000 fd:00 75 /lib/ld-2.13.so 00a a87000 rw-p 0001d000 fd:00 75 /lib/ld-2.13.so 00a c0c000 r-xp fd: /lib/libc-2.13.so 00c0c000-00c0d p fd: /lib/libc-2.13.so 00c0d000-00c0f000 r--p fd: /lib/libc-2.13.so 00c0f000-00c10000 rw-p fd: /lib/libc-2.13.so 00c c13000 rw-p : c c8c000 r-xp fd: /lib/libgcc_s so.1 00c8c000-00c8d000 rw-p 0001b000 fd: /lib/libgcc_s so r-xp fd: /home/build/programming/main a000 rw-p fd: /home/build/programming/main 0804a b000 rw-p :00 0 [heap] b7fed000-b7fee000 rw-p :00 0 b7fff000-b rw-p :00 0 bffdf000-c rw-p :00 0 [stack] Program received signal SIGABRT, Aborted.

GDB Example application: #include int main(int argc, char* argv[]) { int test=5; void* buf = malloc(16); free(buf); printf("test %d", test); return 0; } Program received signal SIGABRT, Aborted. (gdb) backtrace #0 0x in __kernel_vsyscall () #1 0x00ab42f1 in raise () from /lib/libc.so.6 #2 0x00ab5d5e in abort () from /lib/libc.so.6 #3 0x00af051d in __libc_message () from /lib/libc.so.6 #4 0x00af7fb6 in _int_free () from /lib/libc.so.6 #5 0x d in main (argc=1, argv=0xbffff464) at./main.cpp:9 (gdb) frame #0 0x in __kernel_vsyscall () (gdb) frame 5 #5 0x d in main (argc=1, argv=0xbffff464) at./main.cpp:9 9free(buf); (gdb) list 4int main(int argc, char* argv[]) 5{ 6int test=5; 7void* buf = malloc(16); 8free(buf); 9free(buf); 10printf("test %d", test); 11return 0; 12} (gdb) print test $1 = 5 (gdb)

GDB l = list; b [L] = breakpoint [line]; r = run; bt = backtrace; p= print; n= next; s= step into; c= continue; q= quit; f= frame; gdb./main GNU gdb (GDB) Fedora ( fc14) Copyright (C) 2010 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "i686-redhat-linux-gnu". For bug reporting instructions, please see:... Reading symbols from /home/build/programming/main...done. (gdb) l 1#include 2#include 3 4int main(int argc, char* argv[]) 5{ 6int test=5; 7void* buf = malloc(16); 8free(buf); 9free(buf); 10printf("test %d", test); (gdb) b 7 Breakpoint 1 at 0x : file./main.cpp, line 7. (gdb) r Starting program: /home/build/programming/main Breakpoint 1, main (argc=1, argv=0xbffff464) at./main.cpp:7 7void* buf = malloc(16); Missing separate debuginfos, use: debuginfo-install glibc i686 (gdb) bt #0 main (argc=1, argv=0xbffff464) at./main.cpp:7 (gdb) p test $1 = 5 (gdb) n 8free(buf); (gdb) c What happens next?

Core dump analysis GDB What is it? Core dump/memory dump/system dump – recorded state of the working memory of a computer program at a speciffic time, usually when the program has abnormally terminated. Used to assist error diagnosis in programs, and analyze their specific state.

Core dump analysis GDB When is it useful? Hard to predict, occasional crashes; Inability to reproduce a bug with gdb running; Offline debugging – application/host can restart providing (relatively) continuous runtime; Inability to run gdb or connect via gdbserver; Dynamic memory allocation bugs tracked at the exact moment of faulty memory freeing.

Core dump analysis ulimit – user limits for system- wide resources. -S Change and report the soft limit associated with a resource. -H Change and report the hard limit associated with a resource. -a All current limits are reported. -c The maximum size of core files created. -d The maximum size of a process's data segment. -f The maximum size of files created by the shell(default option) -l The maximum size that may be locked into memory. -m The maximum resident set size. -n The maximum number of open file descriptors. -p The pipe buffer size. -s The maximum stack size. -t The maximum amount of cpu time in seconds. -u The maximum number of processes available to a single user. -v The maximum amount of virtual memory available to the process. programming]#./main *** glibc detected ***./main: double free or corruption (fasttop): 0x08cc7008 *** ======= Backtrace: ========= /lib/libc.so.6[0xaf7fb6]./main[0x804845d] /lib/libc.so.6(__libc_start_main+0xe6)[0xa9fe36]./main[0x ] [...] Aborted (core dumped) programming]# ls main main.cpp programming]# ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 1024 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 1024 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited programming]# ulimit -c unlimited programming]#./main *** glibc detected ***./main: double free or corruption (fasttop): 0x08cc7008 *** ======= Backtrace: ========= /lib/libc.so.6[0xaf7fb6]./main[0x804845d] /lib/libc.so.6(__libc_start_main+0xe6)[0xa9fe36]./main[0x ] [...] Aborted (core dumped) programming]# ls core.2877 main main.cpp

Core dump analysis gdb./app./core – analyze a core dump of a given application. programming]# gdb./main./core.2877 [...] Reading symbols from /home/build/programming/main...done. [New Thread 2877] Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /lib/ld-linux.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/ld-linux.so.2 Reading symbols from /lib/libgcc_s.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/libgcc_s.so.1 Core was generated by `./main'. Program terminated with signal 6, Aborted. #0 0x00dc0416 in __kernel_vsyscall () Missing separate debuginfos, use: debuginfo-install glibc i686 libgcc fc14.i686 (gdb) bt #0 0x00dc0416 in __kernel_vsyscall () #1 0x00ab42f1 in raise () from /lib/libc.so.6 #2 0x00ab5d5e in abort () from /lib/libc.so.6 #3 0x00af051d in __libc_message () from /lib/libc.so.6 #4 0x00af7fb6 in _int_free () from /lib/libc.so.6 #5 0x d in main (argc=1, argv=0xbfb32674) at./main.cpp:9 (gdb) f #0 0x00dc0416 in __kernel_vsyscall () (gdb) f 5 #5 0x d in main (argc=1, argv=0xbfb32674) at./main.cpp:9 9free(buf); (gdb) l 4int main(int argc, char* argv[]) 5{ 6int test=5; 7void* buf = malloc(16); 8free(buf); 9free(buf); 10printf("test %d", test); 11return 0; 12} (gdb) p test $1 = 5

GDBServer GDB Makes it possible to remotely debug programs – no need to have sources and gdb on the target machine – be it a STB, TV, or a mobile phone. Connection can be established via serial lane/TCP. GDBServer has to be run on target machine: gdbserver :2159 hello_world GDB example, on other machine: gdb hello_world target remote :2159 Rest of the debugging process proceeds in an ordinary way.

GDB in Eclipse GDB GDB can be fully wrapped into GUI (simmilar to Visual Studio) using Eclipse IDE.

GDB in Eclipse GDB

GDB in Eclipse GDB

GDB in Eclipse GDB

Cross compilation

Cross compilation – what is it? Creating executable code for a platform other than one on which the cross compiler is running. Example: Compiling executable code for an Android smartphone from a Windows OS, using a cross compiler. Cross compilation

When is it useful? Embedded computers development – limited resources, often unable to run a compiler, have a file system, or a development environment – for example a microwave oven; Debugging and testing may require more resources than available on the target; Multiple target compilation – when creating an application for several OS, all of the versions can be build in one environment; New platform development – creating the minimal ammount of necessary tools, for example bootstrapping – writing a self-hosting compiler. Cross compilation

Other approaches Virtual machines (such as Java’s JVM) – allows the same compiler output to be used across multiple target systems. They are however often slower. Cross compilation

Thank You!