Tag: coccigrep

Coccigrep improved func operation

Coccigrep 1.11 is now available and mainly features some improvements related to the func search. The func operation can be used to search when a structure is used as argument of a function. For example, to search where the Packet structures are freed inside Suricata project, one can run:

$ coccigrep -t Packet -a "SCFree" -o func src/
src/alert-unified2-alert.c:1156 (Packet *p):         SCFree(p);
src/alert-unified2-alert.c:1161 (Packet *p):         SCFree(p);
...
src/alert-unified2-alert.c:1368 (Packet *pkt):         SCFree(pkt);

With coccigrep 1.11, it is now possible to look for a function with a regular expression. For example, to see how a time_t is used in the print function of Suricata which are all starting by SCLog (SCLogDebug, SCLogWarning, …), you can simply run:

$ coccigrep -t time_t -a "SCLog.*" -o func src/ 
src/defrag.c:480 (time_t *dc->timeout):     SCLogDebug("\tTimeout: %"PRIuMAX, (uintmax_t)dc->timeout);

With 1.11 version, the func operation is now more useful. It is also more accurate as casted parameters or direct use of a structure (instead of usage though a pointer) are now supported.

Set or unset define variables in Coccigrep

Following a discussion with the great Julia Lawall, she added a new feature in coccinelle: it is now possible to define as set or unset some variables. This option has been added in coccigrep 1.9 and requires coccinelle 1.0-rc14.

For example, let’s have a code like Suricata where a lot of unit tests are implemented. The structure of the code is the following:

REGULAR CODE

#ifdef UNITTESTS
 TEST CODE
#endif

When doing search in the regular code, you don’t want to be bothered by results found in the test code. To obtain this result, you can pass the -U UNITTESTS option to coccigrep to tell him to consider UNITTESTS variable as undefined. If you want to define a variable, you can use the -D flag.

If you are using coccigrep inside vim, you can set the coccigrep_path variable with this option. The basic vim syntax is:

let g:coccigrep_path="coccigrep -U UNITTESTS"

As I wanted to have it for all query in my Suricata source directory, I’ve added at the end of my ~/.vim/after/syntax/suricata.vim file:

autocmd BufEnter,BufNewFile,BufRead */git/oisf/* let g:coccigrep_path="coccigrep -U UNITTESTS"

What’s new in coccigrep 1.6?

I did not write any article on coccigrep since the 1.0 release. Here is an update on what has been added to the software since that release.

C++ support

Coccinelle has a basic C++ support which can be activated by using the –cpp flag in coccigrep.

Patches information

The -L -v options on command line will display a description of the match available on the system.

$ coccigrep -L -v
set: Search where a given attribute of structure 'type' is set
 * Confidence: 80%
 * Author: Eric Leblond 
 * Arguments: type, attribute
 * Revision: 2

For the developer, this is obtained from structured comments put at the start of the cocci file:

$ head src/data/set.cocci 
// Author: Eric Leblond 
// Desc: Search where a given attribute of structure 'type' is set
// Confidence: 80%
// Arguments: type, attribute
// Revision: 2
@init@

This is thus an easy way to document the search operation. Please note, that this will also work for the operations put in the user or system custom directory.

Context line display improvement

Guillaume Nault has contributed a series of patches that greatly improved the display of context lines:

$ coccigrep -C 3 -t Packet -a flags -o set decode*c 
decode.c-90            -     }
decode.c-91            - 
decode.c-92            -     PACKET_INITIALIZE(p);
decode.c:93 (Packet *p):     p->flags |= PKT_ALLOC;
decode.c-94            - 
decode.c-95            -     SCLogDebug("allocated a new packet only using alloc...");
decode.c-96            -

Documentation

A man page is now available.

Python 2.5 support

The 1.6 release came with a code modification that permit coccigrep to run with python 2.5. Some users seem to still use this old version of Python and the support was not requiring to degrade coccigrep code. It has even improved it.

Option to read file lists from a file

Thomas Graf has contributed the -l option which provides a way to specify a file containing the list of the files to search in.

Operation improvement

The set operation has been improved and is now more accurate thanks to the support of all related operators.

Conclusion

Coccigrep is becoming more and more mature over time. The existing code base remains and a polishing work is currently under progress. One last point on the project is that some Linux and *BSD distribution seems to have done packages. This is the case of Aur, Gentoo, NetBSD, OpenBSD, Mandriva and soon Debian if the intention to package is confirmed.

Eric Leblond: Introduction to coccinelle

The Netfilter workshop being a developer conference, I’ve decided to presente an introduction to the coccinelle tool. Coccinelle is a program matching and transformation engine for the C language which is used in many place and among them in the Linux kernel. It is able to perform C clever modification in the code. If you ever had to modify multiple code files following an API change, I invite you to have a look at the slides or my Coccinelle for the newbie page. I’ve also presented my coccigrep tool which is a easy to use semantic grep.

The slides are available: nfws_coccinelle

Coccigrep, a semantic grep for the C language

Introduction

When diving in some code with a relative important size, I’ve often ask myself: where is this attribute used for this structure ? Where it is set ? Using grep is not a good answer to theses questions: you can’t guess the name of the variable of a given type and even an attribute name can be shared between multiple structures. I was in need of a semantic grep!

Having played a bit with coccinelle, I did know that it could be used with success to build a semantic grep for the C language. But coccinelle syntax is not easy and I decided to build something with a simplistic syntax. Something that would integrate easily into vim to be able to use it like grep. I’ve launched my preferred editor and this something is now called coccigrep.

Meet the beast

The syntax is trivial, to find where the datalink attribute of the Packet structure is set in all source*c file:

$ coccigrep  -t Packet -a datalink -o set src/source*c   
src/source-af-packet.c:300 (Packet *p):     p->datalink = ptv->datalink;
src/source-erf-dag.c:525 (Packet *pkt):     pkt->datalink = LINKTYPE_ETHERNET;

You can use coccigrep from vim. If you run in vim:

:Coccigrep -t Packet -a datalink source-*.c

The matches will appear in the quickfix list and the file corresponding to first match will be opened at the corresponding line. Note that you can use completion on structure and attribute names based on tags (generated by :make tags).

Coccigrep supports syntax highlighting through the pygments module. For example, running coccigrep -t Packet -a datalink -o test -c -A 3 -B 3 -f html /tmp/test.c will output to stdout some colorized HTML code:

/tmp/test.c: l.300 -3, l.300 +3, Packet *p
        hdrp->sll_protocol = from.sll_protocol;
    }

    while (p->datalink >= ptv->datalink) {
    	SET_PKT_LEN(p, caplen + offset);
    	if (PacketCopyData(p, ptv->data, GET_PKT_LEN(p)) == -1) {
           TmqhOutputPacketpool(ptv->tv, p);

More information and download are available from coccigrep page.