Archive for December, 2012

And here is the difference!

Posted: December 30, 2012 in Work

So finally over and out call to python this time. Everything was going fine but then suddenly a loop that should take not more than few second (2-5 seconds ) was taking around 5 minutes. Even after highly optimizing it, the time could not be reduced to few seconds as I mentioned. So after hours of struggling and debugging Machine Vision department decided to stick to C/C++.

So yesterday, I started once again  from scratch, this time in C++. Surely the day was not good (rather night I must say). I started in even. I wrote code to find distance contrast with is the image using histogram over HSV range. And I got some strange outputs. And again the debugging process started. After rechecking each thing , handling data types properly I got nothing. So after hours of searching and debugging, I gave a Mayday Mayday call to my senior. It might be around 0.00 am when I asked for his suggestions. I must say I have a very nice senior (Mr. Harsh Agrawal). He helps me every time I give away a Mayday call. So it started around 0.00 am. We both went through code again and again. Then following the Senior’s advice I started comparing all values in all matrices. Printed almost all matrices and compared them, rechecked all calculations. And this continued till 4 am , when I figured out that I used a wrong flag in one of the normalization function. I was happy that ultimately problem was resolved, but a silly mistake and few wrong comparisons(manually) wasted 6-7 hours. So finally I am able to calculate distance contrast perfectly. And am really thankful to Harsh Agrawal for his support.

The day was going still better today and then I started using cvBlob library. I have been coding in Opencv c++ so I read images as Mat format. But cvBlob library was written in C which used IplImage data structure. It is easy to convert an IplImage data type to Mat but I needed to do just the opposite.

was actually using cvLabel fucntion. For it i needed an image to be converted into IplImage format.I  tried following commant

IplImage result_image = result;

which indeed worked perfectly for me.

Though today I had to spend some of my time debugging, but I gave no Mayday calls and were able to solve all issues with in time constrains.

Nothing much to right today. Team is planning alot for future. Lots of planning means lots of changes. It even means the change in my plan.

Till now I am successful in finding blobs.


Histogram exam and my paining arm

Posted: December 25, 2012 in Work
Tags: , ,

I was thinking to find histogram over the HSV ranges in an image that too by using python. Yes, I have done this many times using OpenCV functions. But I am trying to search if I could replace a standard hist in OpenCV with numpy if possible.
Normally, using createHist() function requires size of bins , dimension and the ranges to create a histogram object. The same can be done by using numpy to create a multidimensional array of the same dimensions. After a lot of brainstorming i just 4 lines of code to find histogram. This proves me correct that I can code in any famous computer language provided I am given INTERNET.

So this is how i did it. I do not want to display the histogram, so I focused on formulating array (numpy) to store the histogram and eliminating the overheads in displaying them. I wanted to find histogram over just Hue and Sat range from HSV form. Since OpenCV allows us to specify the channel, this prevented me from splitting the image into three respective channels. Then i first calculated a Histogram for Hue range and normalized it. After that i followed same procedure for S range. I first calculated hist and the normalized it. Merging both of then in different dimensions of a single array gave me the required result.

The outcome seems simple, but i would just say  “mission accomplished !”

I thought the above would work. but it did not . So i struggled more to optimize my code rather to make it work properly. But after lots of struggling i got error. Debug too failed. Python is a good wrap up language. But i dont know why was i not able to debug it properly. May be because of run time nature of opencv.

But after lots of searching i found the solution. And I also found that python is smarter, and has a beauty in it. OpenCV needs to properly document its python implementation.So finally Mission accomplished!

Aim is to perform some initial steps involved in mean shift segmentation of an image. To recognize the objects in image i first want to remove the texture from image to have effective segmentation.  After performing mean shift filtering we get filtered “posterized” image with color gradient and fine-grain texture flattened.

In opencv mean shift filtering can be implemented on the image by using function PyrMeanShiftFiltering(). Apart from source and destination image, it takes radius of spatial window, radius of colour window as its parameters. There are few more default parameters that are implementation dependent.

At every pixel (X,Y) of the input image (or down-sized input image) the function cv2::PyrMeanShiftFiltering  in openCV executes meanshift iterations, that is, the pixel (X,Y) neighborhood in the joint space-color hyperspace is considered:

(x,y): X- \texttt{sp} \le x  \le X+ \texttt{sp} , Y- \texttt{sp} \le y  \le Y+ \texttt{sp} , ||(R,G,B)-(r,g,b)||   \le \texttt{sr}

where (R,G,B) and (r,g,b) are the vectors of color components at (X,Y) and (x,y), respectively (though, the algorithm does not depend on the color space used, so any 3-component color space can be used instead). Over the neighborhood the average spatial value (X',Y') and average color vector (R',G',B') are found and they act as the neighborhood center on the next iteration:

(X,Y)~(X',Y'), (R,G,B)~(R',G',B').

After the iterations over, the color components of the initial pixel (that is, the pixel from where the iterations started) are set to the final value (average color at the last iteration)

Now the point is that I have to perform this function on an image that has been obtained by performing pyrdown three time. But at the same time i do not want to lose any info when I pyrup this image after performing meanshiftfiltering.  By performing filtering I get the image below

But the cv::PyrMeanShiftFiltering also has a parameter maxLevel (default value = 0 ). When maxLevel > 0, the gaussian pyramid of maxLevel+1 levels is built, and the above procedure is run on the smallest layer first. After that, the results are propagated to the larger layer and the iterations are run again only on those pixels where the layer colors differ by more than sr from the lower-resolution layer of the pyramid. That makes boundaries of color regions sharper.

Python bit Eclipse

Posted: December 22, 2012 in Kreeps
Tags: , , , ,

This post is dedicated to my dear juniors.
Today i linked Eclipse on Linux with Pydev (Python Development Plugin). A program in python can be written in even a notepad, but investing time in setting up a proper IDE is fruitfull.

Follow following steps to setup python on your Eclipse IDE

The PyDev Ubuntu setup requires some configuration in Eclipse. Installing PyDev on Ubuntu is similar to the installation on Windows or Macintosh.

1. Run Eclipse and then go to Help | Software Updates | Find and Install…

2. In the Feature Updates window, select Search for new features to install and click Next.

3. In the Install window, select New Remote Site.

4. In the New Update Site window, specify the following:

Name: PyDev

The old URL was

5. In the Search Results window, select both PyDev Extensions and click Next.

6. In the Feature License window, agree to the license agreement and click Next.

7. Configure the Python interpreter by going to Window | Preferences | Pydev | Interpreter – Python.

8. In the Python interpreters section, click New.

9. Locate your Python interpreter (e.g., /usr/bin/python).

10. When prompted, select your System PYTHONPATH. You can just leave the default checkboxes selected and click OK.

Now I assume the reader already has python OpenCV installed
SO follow these two steps now :
Add /usr/local/lib/python2.7/dist-packages to the System Libs
Add cv to the Forced builtins
All set now

Happy Image processing !

I found Python today!

Posted: December 21, 2012 in Kreeps
Tags: ,

I have been coding in C and C++ for all my projects related to OpenCV till now. But when it comes Machine Vision project in team UAS-DTU (code name: Shaurya) we need to consider every possible parameter. Even if it means shifting to our code to Python if it can make it more efficient. So Machine Vision team has started to take Python into consideration. Python is a general-purpose, interpreted high-level programming language whose design philosophy emphasizes code readability. Opencv’s Python language is an implementation using SWIG. SWIG is a software development tool that connects programs written in C and C++ with a variety of high-level programming languages. OpenCV in python seems to make algorithm implementation faster. But i don’t think it will affect the overall processing time since, everything is ultimately processed in c/c++ as Python is just like a wrapper. But still it is fair to Python a chance since even if nothing it could possibly reduce coding/implementation time. It may improve efficiency. Obviously I googled to find if OpenCV’s Python implementation is good enough to port our code to it. But I could not find any major advantage. So now I decide to have hands over it myself.

Currently we are using canon G10 for image acquisition. Obviously the decision of using it was good enough. Canon stopped manufacturing G10 even before we selected that camera. To make things (the Machine Vision code especially) better, I thought to change certain things. Now when the airframe is being restructured, I am thinking to change the camera. If the size, weight and obviously the budget allows I can think over to shift to a DSLR. Selecting an appropriate camera is important because if the acquired image is not good enough then obviously the image processing code will lead to false or no results.
So to select camera, I first short listed few cameras. I first prepared the list of cameras that may be given special attention because they have already been used for similar purpose like ours by some other people. Secondly I searched for all cameras that are supported by gphoto2 library (Canon G10 is one of them). Thousands of cameras were listed in this list. Then I started eliminating all point and shoot cameras. Obviously my aim was to select the best DSLR with in our budget that satisfies all our requirements. I already have been using a point and shoot camera (G10) that apparently satisfies our entire requirements. So now to make significant improvement in the image I need to consider the difference in a point and shoot and DSLR cameras. Moreover due to weight, economical and quality constrains I was made to focus only on Nikon and Canon cameras. This is how I shortlisted my long list of cameras. Obviously I knew what all parameters I am looking for within these cameras.

Cameras are framing systems which acquire a near-instantaneous “snapshot” of an area (A), of the surface. Camera systems are passive optical sensors that use a lens (B) (or system of lenses collectively referred to as the optics) to form an image at the focal plane (C), the plane at which an image is sharply defined.

For efficient processing I need a detailed image of the ground. The ground coverage of a photo depends on several factors, including the focal length of the lens, the platform altitude, and the format and size of the film. The focal length effectively controls the angular field of view of the lens and determines the area “seen” by the camera. The longer the focal length, the smaller the area covered on the ground, but with greater detail (i.e. Larger scale). Since the plane is moving with high speed, I need to find the optimum focal length so that I do not miss any land in continuous images and image still remains detailed as much possible. A zoom lens of focal 18-55mm with APS-C camera was found appropriate.

The area covered also depends on the altitude of the platform. At high altitudes, a camera will “see” a larger area on the ground than at lower altitudes, but with reduced detail (i.e. Smaller scale). In my case altitude may vary from 100 ft to 700 ft.
Also I need a fairly good shutter speed so that there is almost 10-50 percent land overlaps in my consecutive images. Using G10 now delivers one image in almost 3 seconds. Moreover I need very fast shutter speeds in order to combat the movement and vibrations of the aircraft. I can achieve these speeds with a combination of wide aperture and ISO. While there are many ways to set the camera, my suggestion is using Aperture Priority mode. Setting the aperture to one of the widest settings allows plenty of light into the camera. Depth-of-field is not an issue since most everything will be focused at infinity.
Depending on the brightness of the day, I start with the ISO at 100 or 200 and check the shutter speeds that are achieved with that aperture / ISO combination. If the shutter speed is below the range, I raise the ISO until it is in that range.

Now days, everyone with a digital camera considers himself as a professional. No doubt I also have a love for photography. So I thought putting something about sensors used in cameras on the blog. Most digital (rather almost all) use sensors from CMOS family or CCD family.
Both the technologies keep on improving. CCD (charged coupled device) may be better from CMOS (complementary metal oxide semiconductor) in one aspect while CMOS might be better in another. Each has unique strength and weaknesses.
Both these sensors convert light into electric charge. In a CCD, every pixel is transferred through a very limited number of output nodes (often just one) to be converted to voltage buffered, and sent off-chop as an analog signal. All of the pixels can be devoted to light capture, and the output’s uniformity is high. In CMOS, each pixel has its own charge-to-voltage conversion, and the sensor often also includes amplifiers, noise-correction, and digitization circuits, so that the chip outputs digital bits. With each pixel doing its own conversion uniformity is lower.

But the point why I am concerned about the sensor is little application dependent. Let’s understand it. Now suppose I have a camera on a plane taking images of propeller regularly. Both kinds of sensors produce different types of distortions or artifacts when they click images of rotating propeller.
In one of the form, the image contains curly rippling lines while in the other it produces a sort of a propeller which keeps moving in consecutive frames. Curvy image is generally produced by CMOS sensors. While the other distortion is produced by CCD sensors.

These artifacts are majorly due to different e shutter methods used. CMOS sensor uses a rolling e-shutter while a CCD sensor uses an uniform e-shutter. To get a better image fps could be increased. It is seen that at a particular fps rate, sensor produces images showing stable propeller even though it is rotating. SO now i know what I am going to lose (or gain) if i use a CMOS based camera with low fps rate.
So this was propeller distortion. This is a major concern while talking about the aerial image acquisition.

Certainly Natural User Interface remains incomplete if a smart system can not recognize common human gestures. I wondered if my computer could tell me what action i am performing (like walking, running etc). So i planned to study kind of activity recognition. According to some source on internet “Activity Recognition aims to recognize the actions and goals of one or more agents from a series of observations on the agents’ actions and the environmental conditions.”

Like all most all other Image Processing stuff that involve recognizing stuff from an image, activity recognition also follows three steps:

1) Detection/Segmentation : It is the process in through which we separate required portion (or what I call Object of Concern) from Image. In my case OoC is the human body. I separated (or segmented) this from the image simply using background and foreground subtraction.

2) Classification: This process involves processing the segmented frames using certain parameters or matching with some templates

3) This part majorly deals with reasoning engines. They encode the activity semantics based on the lower level action primitives.


Posted: December 10, 2012 in General
Tags: ,

Aarush X1 is the result of an industry-academia collaboration between Lockheed Martin and
Delhi Technological University. The Unmanned Aerial System has been designed and fabricated
by a group of undergraduate students of the university, who received timely guidance from
mentors in the faculty and Lockheed Martin. The UAS has been designed according to the
constraints imposed by urban environments, making the system invaluable for ISR in cities like
Delhi and Mumbai. The system, in fact, can be modified to adapt to specific mission-dependent
Aarush X1

The UAS was unveiled by Dr. V.K.Saraswat, Director General, D.R.D.O, on November 2nd,
2012, at the Lokarpan Ceremony held at the India Habitat Centre. The ceremony was presided
over by Dr. D.N. Reddy (title), and Dr.R.P.Aggarwal. The event witnessed a small presentation
by the student project manger and team captain, outlining the design and fabrication process
adopted by the team. Mr. John Sheehan, who along with Dr. Peter Drewes, mentored the students,
highlighted the key accomplishments of the project, and congratulated the students on their
acheivement.Mr. Abhay Paranjape, (title) also attended the ceremomy, and emphasised on the
importance of the sustenance of student effort, and long term relationships between industry and
academia , in his talk.

Dr. V.K.Saraswat lauded the student effort and expressed a desire to nurture the student talent. He
also outlined some major capabilities that are demanded of current Unmanned Air Systems, like
stealth and payload compatibility.The Lokarpan Ceremony was also attended by representatives
from HAL, NTRO, Army, IAF and DST.

The ceremony concluded with an invitation to all the attendees to witness the First Public Flight
of Aarush-X1, which was to be held the following day (November 3rd) at Karnal Institute of
Civil Aviation, Haryana.

The Flight Day was attended by engineers from HAL and NTRO amongst others.
The UAS did a single sortie for about 3 minutes. Telemetry and real time video, were displayed
at the portable ground station. The UAS was taxied to the viewing area, after the landing, for a
visual inspection by the attendees, which was followed by a question and answer session.

Courtesy: Bhavya N. Gupta