I can direct assign value to a variable in my bash shell why I still need the command ‘export’ ? What is the different between them?

If you have doubts about this, please continue to read on it.

Lets do an experiment in a bash shell:

$ HELLO=world
$ export HELLO2=world2
$ echo $HELLO $HELLO2
world world2

Hey, it works exactly the same!

Not really exactly the same if you use it in your bash script. Let me show you an example. Let say I have a bash script main.sh , which it calls child.sh.

child.sh

#!/bin/bash
echo $HELLO $HELLO2

main.sh

#!/bin/bash
HELLO=world
export HELLO2=world2
./child.sh

Now lets execute ./main.sh

$ ./main.sh
world2

Hey! the ‘world’ has gone!

When we assign the variable directly, we make the variable local, when we call a child script, the local variable will not available for the child script. In contrast, if we export the variable, we make it appear as an environment variable, which it will be available globally within that bash session. That is the reason, child.sh reads $HELLO2 but not $HELLO.

Sometime you may wish to create a bash script to do something based on a config file, what actually was in your mind? How to implement the config file for your bash script?

First I think of using command ‘cut’, where I can create a list of tag=value, then uses ‘cut‘ to read out the tag and value. But that is so wrong, because I could just using command ‘source’ to do it instead.

Let say I have a config.ini

PATTERN="$hello*"
FILENAME=hello.txt
OUTPUT=result.txt

And I want my bash_script to read these variables and perform grep.

#!/bin/bash
source config.ini

grep ${PATTERN} ${FILENAME} > ${OUTPUT}

The keyword ‘source’ will import the variable from config.ini into my script, made them available for my use. The source line can be also replace it into this:

. config.ini

DOT(.) is equivalent to command ‘source’.

I can also source another bash script which contains if-else logics, the same way I source my config.ini.

But bare in mind, if your target source file contains ‘exit’, it will exit your parent script.

For example, I have a parent.sh sourcing config.sh, where my config.sh contain exit.

config.sh

#!/bin/bash
HELLO=world
exit

parent.sh

#!/bin/bash
source config.sh
echo ${HELLO}

When I try to execute the ./parent.sh, my script immediately exit before it execute the ‘echo’.

I hope that this post will gives you an idea of using command ‘source’ to implement your config file for your bash script.

You may already realized Linux happened to appear at many places, such as web server, storage server, desktop, kiosk machine, mobile devices. Yes, more and more devices running embedded Linux. Yeah, Android is a modified version of Linux kernel too!

Scarcity is still an issue, embedded Linux can be very different to Linux that hosted on desktop or high-end servers. You may be provided with simple shell (sh but not bash, not ksh and not csh) and limited commands, it bundles into a BusyBox, The Swiss Army Knife of Embedded Linux.

I am a bit upset when I holding this so called Swiss Army Knife ( probably a customized one, super tiny one), why this command also don’t have, that command also not available. Well, I am still quite new to embedded Linux perhaps.

My busybox does not have bash, therefore a lots of bash syntax can’t be applied to my script, ouch! I do wrote some post about bash. Such as if … then … fi and download multiple files from a site using for loop like c in bash. The if…then…fi examples may still valid for shell script, because that is the fundamental one.

The very basic for loop for shell script is like this

for p in 1 2 3 4 5
do
     echo $p
done

You can do simplified it by using command seq ( if seq is available in your busybox)

for p in `seq 1 5`
do
     echo $p
done

Well, my swiss army knife doesn’t contain me seq command, it doesn’t have bash, no top command, giving me a fake ps with limited info. I can’t check my how busy is my processors and thread easily (because fake ps do not tells you that), I need to access it through /proc/[PID] and thank God, it does provide me the cut command.

Yes I able to cut by using my swiss army knife.

Due to the scarcity, I learn more about /proc. /proc is a pseudo-filesystem that provides tones of process information. Check out the proc 5 manual for detail information.

man 5 proc

What I am interested is the file /proc/[PID]/stat, the status information about the process. stat provides a long array of values, I am interested on column 1,2,14 and 15 which is PID, Command, utime and stime.

Ok, list me all the process’s utime and stime. I wrote a simple script call probe.sh.

#!/bin/sh
for p in /proc/[0-9]*;
do
    cat $p/stat | cut -d" " -f1,2,14,15
done

The output looks decent, time takes less then 1 second.

...
6 (cpuset) 0 0
608 (scsi_eh_0) 0 1
612 (scsi_eh_1) 0 0
615 (scsi_eh_2) 0 1
661 (kpsmoused) 0 0
671 (kstriped) 0 0
677 (kondemand/0) 0 0
7 (khelper) 0 8
707 (usbhid_resumer) 0 0
854 (udevd) 0 11

real    0m0.965s
user    0m0.138s
sys     0m0.752s

But the list is long and overwhelming. What if I have targeted pid range?

#!/bin/sh
START=2000
END=3000
until [ $START -gt $END ]
do
    p="/proc/$START/stat"
    if [ -f $p ] ;
    then
        cat $p | cut -d" " -f1,2,14,15
    fi
    START=`expr $START + 1`
done

The output is promising, but it took too long to produce that.

2391 (syslog-ng) 0 0
2392 (syslog-ng) 1 11
2451 (gpm) 0 0
2882 (dhcpcd) 0 0

real    0m7.145s
user    0m0.959s
sys     0m5.640s

The expr for arithmetic function seems to consume a lots of processing time.

Another approach, which amazingly gives correct result and fast.

START=2000
END=3000
for p in /proc/[0-9]*;
do
    pid=`basename $p`
    # omg no basename? replace with the line below
    # pid=`cat $p | cut -d"/" -f3`
    if [ $pid -ge $START ]
    then
        if [ $pid -le $END ]
        then
            cat $p/stat | cut -d" " -f1,2,14,15
        fi
    fi
done

Check out the result. Outstanding isn’t it?

2391 (syslog-ng) 0 0
2392 (syslog-ng) 1 11
2451 (gpm) 0 0
2882 (dhcpcd) 0 0

real    0m0.550s
user    0m0.069s
sys     0m0.447s

I had briefly illustrate the looping mechanism on shell script, I hope that gives you some ideas while you working under embedded Linux.

Usually we start our virtual machine by clicking through the GUI. If your guest machine is a server, it would be making more sense to be run as background process.

Well, my case, I have ‘gentoo’ installed in virtual box, which expose to outside world through ssh. I would like it to start as background process, how can I do that?

Virtual Box allows you to start your virtual machine through command line VBoxHeadless, by adding ‘&’, you can make the process run as background.

VBoxHeadless -s [guestname] &

I wrote a bash script to help myself perform the background startup, reboot, poweroff and status query. Assume my guest machine name as ‘gentoo’, I wrote a script name as ‘gentoo’. Now I can do ‘gentoo start’ to start it at background.

#!/bin/bash

if [[ -z "$1" ]]
then
    echo "Usage:"
    echo "      gentoo [status|start|reboot|poweroff]"
    exit
fi

if [[ $1 == "status" ]]
then
    # print out the machine state
    VBoxManage showvminfo gentoo | grep State
elif [[ $1 == "start" ]]
then
    # start without vrdp
    VBoxHeadless -s gentoo --vrdp=off &
elif [[ $1 == "reboot" ]]
then
    # send ctrl+alt+del to guest machine
    VBoxManage.exe controlvm gentoo keyboardputscancode 1d 38 53 b8 9d
elif [[ $1 == "poweroff" ]]
then
    # simulate pressing power off button, will power cut immediately
    VBoxManage controlvm gentoo poweroff
fi

With the capabilities of command line to manipulates the guest machine, it helps to support for more customization and automation. This is the main reason I like about VirtualBox.

Have fun :)

If you came across the solution of setting up port forwarding for ssh port like this:

VBoxManage setextradata [guestname] "VBoxInternal/Devices/pcnet/0/LUN#0/Config/ssh/HostPort" 2222
VBoxManage setextradata [guestname] "VBoxInternal/Devices/pcnet/0/LUN#0/Config/ssh/GuestPort" 22
VBoxManage setextradata [guestname] "VBoxInternal/Devices/pcnet/0/LUN#0/Config/ssh/Protocol" TCP

You may want to try out this alternative approach. This approach is using the same command ‘VBoxManage’, but it is more flexible and simple.

1. First of all, I power off my guest machine (virtual machine), this approach refuse to work if my guest machine is running.

2. Next, I query my guest machine’s vminfo.

Let say my guest machine ‘s name is ‘gentoo’ (I am gonna use this as example for the entire post) , I will do this:

VBoxManage showvminfo gentoo

It shows a long list of very informative results. I am only interested on my guest machine’s NIC info. Therefore I do this instead.

VBoxManage showvminfo gentoo | grep NIC

NIC 1:           MAC: 0800272DC585, Attachment: NAT, Cable connected: on, Trace: off (file: none), Type: virtio, Reported speed: 0 Mbps, Boot priority: 0
NIC 1 Settings:  MTU: 0, Socket( send: 64, receive: 64), TCP Window( send:64, receive: 64)
NIC 2:           disabled
NIC 3:           disabled
NIC 4:           disabled
NIC 5:           disabled
NIC 6:           disabled
NIC 7:           disabled
NIC 8:           disabled

Based on the result of vminfo, indicates that my guest machine configured a NIC which is NIC1.

Ok, now lets do a port forwarding. Take ssh port as an example. I would like to forward my guest ssh port (22) to my host machine at port (2222). I will do this:

VBoxManage modifyvm gentoo --natpf1 "ssh,tcp,,2222,,22"

The syntax for modifyvm –natpf shows as below:

VBoxManage [guestname] --natpf[1-N] "[function-name],tcp|udp,[hostip],[hostport],[guestip], [guestport]" 

‘function-name’ can be any name which well describe the port forwarding rule.

Lets querry the vminfo again.

VBoxManage showvminfo gentoo | grep "NIC 1"

NIC 1:           MAC: 0800272DC585, Attachment: NAT, Cable connected: on, Trace: off (file: none), Type: virtio, Reported speed: 0 Mbps, Boot priority: 0
NIC 1 Settings:  MTU: 0, Socket( send: 64, receive: 64), TCP Window( send:64, receive: 64)
NIC 1 Rule(0):   name = ssh, protocol = tcp, host ip = , host port = 2222, guest ip = , guest port = 22

Observed that I ignore the IP address settings because I am using DHCP on my guest machine NIC.

To delete the port forward rule, I do this:

VBoxManage [guestname] --natpf[1-N] delete [function-name]: 

VBoxManage gentoo --natpf1 delete ssh

At last, I boot up my guest machine to verify the port forwarding setting.

VBoxManage startvm gentoo

VBoxManage can do more than just port forwarding, check it out the manual if you want to know more.

Hope you enjoy.

I am curious about what are Linux kernel do? Many years ago, due to the tedious steps to compile the kernel, I give up dive into it at the very beginning stage, which is compiling kernel. But recently this urge come back to me. I started to wonder whether is it exist a Linux distro which provides kernel build environment.

I came across a forum (lost track of the link), someone mention in fact Gentoo is a good candidate for the kernel build. As simple as emerge the kernel source, and you will have the latest kernel source code under /usr/src/linux.

emerge gentoo-sources

Installing Gentoo is not like installing Ubuntu/Fedora/Suse, where you can clicks the way and get a fancy desktop. I don’t want a fancy Gnome/Kde, what I want is a text console, which allows me to learn and hack the kernels, therefore Gentoo came as a charm for me.

Gentoo Linux is a distro that gives you the best optimized system, best performance. It sounds like the best distro for you isn’t it? The reason why Gentoo can be so optimized and performance wise competent because you have to compile everything including kernel. emerge is a command that helps you download the source codes, resolve dependencies, compile and install it right for you.

The challenging build steps comes with a very details installation guide, you can start with it by reading Gentoo handbook.

I create an image on VirtualBox, network setting attach to NAT. under Advanced option Adapter type set as virtio-net. The reason I choose virtio-net as the adapter type, because that is the only Ethernet kernel module available on the Gentoo minimum live CD which VirtualBox (version 3.2.2) could provide.

I need to enable virtio as kernel module in kernel config, refers to the live cd, there are few kernel modules to enable. virtio, virtio_ring, virtio_net and virtio_pci.

In order to have a broaden view on console, I enable uvesafb. You can follow the guide here. It giving me problems that, no matter how I set my video option, I can only see half of the screen> Fortunately, I found some guides for Gentoo under VirtualBox.

Remove consolefont from boot runlevel

rc-update del consolefont boot

Replace video option:

video=uvesafb:800x600-16,mtrr:4,redraw

For the fully customized OS like Gentoo, it is important to know the various configuration files. Here I wanna highlight some of the important one.

Mount Mapping table: /etc/fstab
This is the common one, to configure the mounting table, you have to edit this file.

Grub Bootloader config: /boot/grub/grub.conf
This is where you adding your kernel entries as your boot option. Video option is done here too.

Make config: /etc/make.conf
Installing package means compiling, to optimizing your build, defines your setting here. Portage related variables can define here too, such as GENTOO_MIRRORS and USE.

Kernel Modules Autoload: /etc/modules.autoload.d/kernel-2.6
You can define your extra kernel modules to autoload while booting up.

List of available kernel module installed.

find /lib/modules// -type f -iname '*.o' -or -iname '*.ko' | less

Network config: /etc/conf.d/net
Define your static IP or dhcp over here.

There are a lots more yet to discover for me, at least for now I am happy with Gentoo ;)

Hope you have fun too.

GNU Global (gtags) is a source code tagging system similar to ctags but more than that. I always use vim as my c/c++ source code editor, and using ctags to helps me travels from function call to function definition. I do write a post indicates how does ctags works with vim.

The reason I shifted to gtags is because ctags is no longer sufficient to me. I have to trace the codes from large volume than last time, and they scatter into many different folders.

Gtags provides me not only travels to definition of the function calls, but also allows me to locate the function calls from the definition, the reverse way. To be precise, it capable of locate object definitions, object references, both of them based on patterns I specified.

Gtags works with vim too, by doing some keymaping tweaks, it works the same like ctags do, and more.

To get gnu global install to your system, check out the official website’s download page.

After the installation, goto your root path of your c/c++ project, and trigger this and go make a coffee.

gtags

Gtags will pickup whatever needed and create FOUR tagging db file, refers here for more info.

In order to allow Gtags working in vim, you have to copy over the gtags.vim into vim plugin’s folder. ( Refers to here on how to install vim’s plugin script.)

Vim with gtags
For vim with ctags, hitting Ctrl+] will jump you to object definition, I gonna do the same for vim with gtags. Besides that, I gonna map Ctrl+\ to list all the object references.

Now, lets us edit vimrc, copy and paste those lines into your vimrc.

"gtags
:map  :Gtags
:map  :Gtags -r
:map  :cclose
:map  :cp
:map  :cn

*nix, vimrc is at ~/.vimrc
windows gvim, Edit > Startup Settings

When Gtags list all the object references, it will open a split windows, to travels from one to another, use :cn and :cp, to close the split windows use :cclose, all are mapped to F11, F12 and F10.

You can find your codes in your shell too, by using command global.

To list all definitions of "GetExecute", you do this:

global GetExecute

For more details listing, you can add in -x option, which it will print out the line as well as the line number.

global -x GetExecute

To list all the references of "GetExamples", you do this:

global -rx GetExamples

You can add in your regex in your search such as *, see this:

global -rx GetLinuxBy*

You can search by symbol as well as grep too.

global -sx GetLinuxBy*
global -gx GetLinuxBy*

And more.

Thanks to Shigio YAMAGUCHI, Hideki IWAMOTO et al. for such wonderful tools and hope you enjoy coding with global.