Preparing IDE and development environment for Zenoss - part1

Finding a good IDE for Zenoss development can sometimes be tricky. Most people resort to use ssh and vim to do the job. Although I like vim a lot, however, it is not the editor of my choice for big jobs. I find a balance of mouse and keyboard short-cuts easier and quicker to use. This is the first part which discusses preparing a VM Dev environment. In part2 I will cover the actual IDE setup.

1- Download the latest VMWare Image from http://community.zenoss.org/docs/DOC-3240?noregister. I am using 32bit 3.1.0 at the time of writing this.
2- I prefer VBox instead of VMWare, so extracted the image, imported in VBox 4.0.12 and added it as a Red Hat Linux guest loading the VMWare harddrive. Then changed the advanced setting of Network interface to use Intel 1000 MT Desktop network adapter instead of PCnet. If you don't do this, your network interface won't come up after boot. After boot login username:root password: zenoss. Remove vmware tools by running vmware-uninstall-tools.pl. Note this step won't be needed if you want to use VMWare. In that case, you just need to run the image and login.
3- Assuming your network works with Internet then run yum groupinstall gnome-desktop
4- Run yum install setuptool system-config-display system-config-network system-config-services system-config-securitylevel gcc make
5- run passwd zenoss to set a password for zenoss user I have put the default 'zenoss'
6- run system-config-display to create initial X config
7- run system-config-securitylevel-tui and then disable SELinux
8- From Vbox menu select install guest additions, this will mount Guest ISO Virtual CD
9- run mount /dev/cdrom /media to mount guest additions
10- Now logout from root and log back in with zenoss user and the password you set
11- run startx, this should bring full gnome desktop which will be in low resolution
12- Open a terminal shell and run following command in it:

[zenoss@localhost ~]$ su - --session-command=/media
                                     /VBOXADDITIONS_4.0.12/VBoxLinuxAdditions.run

This should build whole guest additions successfully. As mentioned if you decided to go with VMWare you won't need to do this step and step 9. I haven't tested that scenario though. After installation you will need to reboot.
13- Login with zenoss user and run startx, now you should have a full blown gnome desktop with all guest VM bells and whistles including shared folders and copy paste.

I personally use a shared folder for project development as I can access source code both from VM and Host OS. To do that you might want to take a look at my older post virtualbox-and-guest-ubuntu-shared

This will continue in part 2...

PS process State codes

PROCESS STATE CODES
Here are the different values that the s, stat and state output specifiers
(header "STAT" or "S") will display to describe the state of a process.
D    Uninterruptible sleep (usually IO)
R    Running or runnable (on run queue)
S    Interruptible sleep (waiting for an event to complete)
T    Stopped, either by a job control signal or because it is being traced.
W    paging (not valid since the 2.6.xx kernel)
X    dead (should never be seen)
Z    Defunct ("zombie") process, terminated but not reaped by its parent.

For BSD formats and when the stat keyword is used, additional characters may
be displayed:
<    high-priority (not nice to other users)
N    low-priority (nice to other users)
L    has pages locked into memory (for real-time and custom IO)
s    is a session leader
l    is multi-threaded (using CLONE_THREAD, like NPTL pthreads do)
+    is in the foreground process group

Debugging a Linux process

To debug a compiled C process in RedHat Linux. You will need to install devel and debuginfo of the package you are insterested in. For example to debug python interpreter you will need to install python, python-debuginfo and python-devel packages. Then find the stuck process PID via PS and then:

# gdp --pid <pid num>

Then you will be in gdp prompt. You then can use following commands:

>continue        will continue execution until you press Ctrl-C
>bt                  show backtrace
>next               execute next line
>list                 show source code
>step               step into child function
>finish              execute until finish the current function
>show help      show extra commands that can be used with show
>print <var>    prints content of a variable

Another good tool for debugging is Valgrind

Co-routines in Python

Co-routines are very interesting subject in python which enables several routines to feed into each other very similar to unix piping. The good point of this approach is that each routine gets executed onces each bit of data is required. So it is very effiecient in terms of CPU and memory usage. Following link is a great resource on this subject.

http://www.dabeaz.com/coroutines/

Postgresql 9.0 and ident

I was trying to create some monitoring tool for new postgresql 9.0 in a project on RHEL. Spent couple of hours tweaking hba file with no effect! Quite disturbing! Then I found the file was somewhere else due to difference between the 9.0 rpm and the original RH one! Here is the command to find it. Quite useful:

postgres=# show hba_file; 
              hba_file             
-------------------------------------
 /var/lib/pgsql/9.0/data/pg_hba.conf
(1 row)

I found it in this blog, quite other nice things in it.

Best wireframe application mockup tool

For a long time, I have been looking for the best and most flexible application UI design tool for rapid prototyping and wireframing. I noticed there weren't any well rounded one available. Either they were too complicated which eventually made me think using a proper IDE like Visual Studio or Ecpilse would be far easier or they lack essential features. For example, they were only for web applications and not desktop or vise versa or didn't support mobile apps or they weren't very expandable. Thus I resorted to use very basic diagramming tool which was actually very effective and more importantly free and opensource called Diagram Designer .
This was all until I found Balsamiq. I must admit this was really great! Far better than what I thought. Very simple, elegant UI, done in Adobe AIR which meant could be run in the browser or outside on all widely used platforms. It supports web apps, desktop, mobile(iphone and android) with loads of control libraries plus a nice community which contribute more controls. Welldone Balsamiq!

Software codemanship

Developing software is an art. Jason from Codemanship have excellent experience and knowledge which tries to share. Following is the list of interesting links related to this matter:

• Crap4J measures CRAP in Java code
• SOLID design
• Dependable Dependencies
• V model testing should be as automated as possible.

Automaticly build VM machines with VirtualBox

There is nice budding project Vagrant which automates the process of creating VMs. I haven't yet experiment with it. But it is an interesting thing to know of.

Juniper JunOS and Cisco IOS emulators

This little beauty http://dynagen.org/tutorial.htm, can run the actual IOS image! and it works fully like a router, routing packets!

The same exist for Juniper http://juniper.cluepon.net/index.php/Olive which can be run even under VMWare http://routerjockey.com/2009/10/03/running-junos-under-vmware/ but these don't include routing engine.

Linux command tips

As I am getting older and my memory sometimes fails embarrassingly, I decided to dedicate this post for some useful bits and peaces here and there which saves a lot of time.

Find
Is an extremely useful utility specially when used with proper piping.

# This will search for the word 'xml' in every file under
# current folder that starts with . (hidden dirs)
kibo@kibo-UbuntuVM:~$ find . -regex '^.\/\..*' -exec grep xml {} \;

# Now to limit the search to directories and only first level
kibo@kibo-UbuntuVM:~$ find . -regex '^.\/\..*' -type d -prune

# To do inverse selection(everything but the ones you selected)
kibo@kibo-UbuntuVM:~$ find . -regex '^.\/\..*' -type d -prune -o -print

# Thus interestingly following prints only files not folders
kibo@kibo-UbuntuVM:~$ find . -type d -o -print

# Search everything except Desktop folder
kibo@kibo-UbuntuVM:~$ find . -path Desktop   -o -print

# Lists all symbolic links from current path downwards
kibo@kibo-UbuntuVM:~$ ls -l `find . -type l`

# List all broken symbolic links from current path downwards
kibo@kibo-UbuntuVM:~$ find -L . -type l

# List all directories with maximum 2 levels of depth
kibo@kibo-UbuntuVM:~$ find . -maxdepth 2 -type d

# Global search and replace! Very handy indeed
kibo@kibo-UbuntuVM:~$ find ./Desktop -type f -exec sed -i 's/hello/goodbye/g' {} \;

Ls
Very basic command but the most useful!

# List all directories nicely!
kibo@kibo-UbuntuVM:~$ ls -ldr */
drwxr-xr-x 2 kibo kibo 4096 2011-04-30 23:36 Videos/
drwxr-xr-x 2 kibo kibo 4096 2011-04-30 23:36 Templates/
drwxr-xr-x 2 kibo kibo 4096 2011-04-30 23:36 Public/
drwxr-xr-x 2 kibo kibo 4096 2011-04-30 23:36 Pictures/
drwxr-xr-x 2 kibo kibo 4096 2011-04-30 23:36 Music/
drwxr-xr-x 2 kibo kibo 4096 2011-04-30 23:36 Downloads/
drwxr-xr-x 3 kibo kibo 4096 2011-05-01 11:02 Documents/
drwxr-xr-x 2 kibo kibo 4096 2011-04-30 23:36 Desktop/
drwxr-xr-x 3 kibo kibo 4096 2011-05-01 11:02 Aptana Studio 3 Workspace/
drwxrwx--- 9 kibo kibo 4096 2011-05-01 21:35 Aptana Studio 3/

Ps
Good during trouble shooting along with top and nmon!

# Lists all python processes with first column(pid)
# and fourth column(cpu time)
kibo@kibo-UbuntuVM:~$ ps fax | grep python | awk {'print $1,$4'}

Rsync
Syncing two directories through SSH

rsync -av -e ssh /blah/*  host105:/opt/blah/blah 

netstat
Lists every single open port based on PID

netstat -taucp

VirtualBox and Guest Ubuntu shared folders

If you have used VirtualBox with host and guest windows, then you know that it has very nice automounting shared folders feature. The same feature is now available for Linux based guests. But there is a hitch. For your user to be able to access the shared folder, you would need to be added to vboxsf group as the permissions don't allow you to see the shared folder. By the way, all shared folders are automounted under /media/sf_<sharedfolder name>.

root@kibo-UbuntuVM:~# ls -l /media/
total 16
drwxrwx--- 1 root vboxsf 16384 2011-04-30 22:59 sf_Shared

root@kibo-UbuntuVM:~# useradd -G vboxsf [your gnome session user]

Or alternatively you can add an existing username in /etc/group file at the end of line which starts with vboxsf.

ensure you logout and login again and voilà!

Python 'with' statement

'with' statement is quite useful nice feature added to python 2.5. Good description is presented at Understanding Python's "with" statement. Following is an useful excerpt:

 
class controlled_execution:
        def __enter__(self):
            set things up
            return thing
        def __exit__(self, type, value, traceback):
            tear things down

    with controlled_execution() as thing:
         some code

or some more tangible example:

>>> with open("my.txt",'w') as f:
...    f.write('hello')
... 
>>> f
<closed file 'my.txt', mode 'w' at 0xb774b2e0>
root@kibo-kubuntu:~# ls
my.txt
root@kibo-kubuntu:~# cat my.txt ; echo
hello 

Nice and easy! huh?!

Antispam for the masses

I recently bumped into a nice project SAUCE. It's daemon sits in front of your good old SMTP daemon (postfix, sendmail,..) and does some aggressive spam assassinations! ;) a good companion for SpamAssassin at the heart of Zimbra!

Python console tab tab autocomplete!

This peace of simple code saves a lot of time! For more info check http://docs.python.org/tutorial/interactive.html

>>> import readline, rlcompleter
>>> readline.parse_and_bind("tab: complete")
>>> import daemon 
>>> daemon.
daemon.__class__(         daemon.__name__           daemon.__subclasshook__(
daemon.__delattr__(       daemon.__new__(           daemon.basic_daemonize(
daemon.__dict__ 
daemon.__doc__            daemon.__reduce__(        daemon.daemonize(
daemon.__file__           daemon.__reduce_ex__(     daemon.errno
daemon.__format__(        daemon.__repr__(
daemon.__getattribute__(  daemon.__setattr__(       daemon.sys
daemon.__hash__(          daemon.__sizeof__(        daemon.writePID(
daemon.__init__(          daemon.__str__(  

If you want to go fancy then try IPython or bPython.

root@kibo-kubuntu:~# apt-get install ipython
root@kibo-kubuntu:~# ipython
Python 2.6.6 (r266:84292, Sep 15 2010, 15:52:39) 
Type "copyright", "credits" or "license" for more information.

IPython 0.10 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object'. ?object also works, ?? prints more.

In [1]: import daemon

In [2]: daemon.
daemon.__builtins__      daemon.__doc__           daemon.__hash__          
daemon.__class__         daemon.__file__          daemon.__init__          
daemon.__reduce__        daemon.__sizeof__        daemon.checkPID          
daemon.sys
daemon.__delattr__       daemon.__format__        daemon.__name__          
daemon.__reduce_ex__     daemon.__str__           daemon.daemonize         
daemon.writePID
daemon.__dict__          daemon.__getattribute__  daemon.__new__           
daemon.__repr__          daemon.__subclasshook__  daemon.errno             

In [2]: daemon.

While you are there, have a look at IPipe which has got some fantastic features.
Even more intresting would be some graphical IDE, isn't it?! then you might want to try wxPython

root@kibo-kubuntu:~# cat /etc/debian_version 
squeeze/sid
root@kibo-kubuntu:~# apt-get install python-wxgtk2.6 python-wxtools wx2.6-i18n
kibo@kibo-kubuntu:~$ pycrust 
PyCrust 0.9.8 - The Flakiest Python Shell
Python 2.6.6 (r266:84292, Sep 15 2010, 15:52:39) 
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
from wxPython.wx import *
/usr/bin/pycrust:1: DeprecationWarning: The wxPython compatibility <snip>
  #!/usr/bin/python2.6
f = wxFrame(None, -1, "Hello world!")
p = wxPanel(f, -1)
b = wxButton(p, -1, "Click me!",(10,10))
f.Show
<bound method Frame.Show of <wx._windows.Frame; <snip>
f.Show()
True 

But fun doesn't end here, the most sophisticated one is PyDev which is an Eclipse plugin.If you don't already know these, then you have plenty to learn. But for an easy ride installation try the latest Aptana 3.0 which comes with PyDev pre-installed plus loads of other goodies. You will need JRE to run it. No installation is necessary just download the package, unzip then run AptanaStudio3. Then you are on your own after that! ;)