Panoramic photograph stitching — again
In an earlier post, I described and implemented a method, that was published recently, to stitch together photographs from a panoramic set. In a comment this morning, Panda asked about the parameters that direct the region merging in the watershed that I used. This set me to think about how much region merging the watershed should do. The only limitation that I can think of, is that we need two regions: one touching the left image and one toughing the right. We can easily do this with a seeded watershed: we create two seeds, one at each end of the region where the stitch should be, and run a seeded watershed. This watershed will not create new regions. You should see it as a region growing algorithm, more than a watershed. However, the regions are grown according to the watershed algorithm: low grey values first. That insures that, when the two regions meet, it happens at a line with high grey values (a “ridge” in the grey-value landscape). The graph cut algorithm can now be left out: the region growing algorithm does everything.
Let’s start with a quick recap. We have two images in variables a and b:
I found the coordinates of a common point in the two images, ca and cb. Then we extend (and crop) the two images so that they have a common coordinate system:
This is where the two methods start to differ. We again look at the difference between the two images in the stitching region, taking the maximum of the differences in the three color channels:
d = max(abs(a-b)); d(0:sz(1)-1+s(1),:) = 0; d(sz(1):imsize(d,1)-1,:) = 0;
The only difference with the d from the earlier blog post is that we do not look at the inverse of the difference, because the region growing algorithm in DIPimage that I will use can grow into high grey values first (i.e. the opposite of what the watershed would do). The next task is to create a seed image. We set the pixels unique to the first image (image a) to 1, and the pixels unique to the second image to 2. The common region, where the stitch will be, we leave 0, so that the two seeds can grow into this area:
c = newim(d,'sint8'); c(0:sz(1)+s(1),:) = 1; c(sz(1)-1:imsize(d,1)-1,:) = 2;
Now we just run the region growing algorithm:
c = dip_growregions(c,d,[],1,0,'high_first'); w = c==2;
Note that we used the 'high_first' option. Using 'low_first' would yield the more traditional seeded watershed. As before, the final composition is trivial using this mask:
out = a; out(w) = b(w);
As can be seen below, the two methods choose a different stitching, but both are quite successful. In this example, the region growing method seems better than the graph cut method, but we’d need to do some more extensive testing to know whether this is always the case or not. These two figures show where the images were stitched with each of the two methods, the graph cut method first, the region growing method second:
Finally, a comparison like in the earlier blog post, comparing the blurring, the graph cut, and the region growing methods:
The new method does not duplicate that one rock in the middle of the zoomed area. This might be coincidence, of course. Don’t assume this method is better just because of this one example!
Feel free to download the script I used to generate the images on this page.
RSS:
Watersheds are minimal graph cuts, in the max norm!
“Some links between min-cuts, optimal spanning forests and watersheds”
Cedric Allene, Jean-Yves Audibert, Michel Couprie, Jean Cousty, Renaud Keriven
Image and Vision Computing (2010)
Thanks Filip!
I was thinking that I should be able to reproduce the watershed result with the graph cut method if I changed the way the weights are calculated…
Hi Cris,
I’ve tried your script. It seems that it will exceeds image boundary since you used a = a(:,0:imsize(a,2)-s(2)-1); Is there any way to solve it? Thanks.
Isabella,
I use DIPimage for the code on this page (and almost everywhere else on this blog). In DIPimage, images have indices starting at 0. You can download DIPimage for free (for non-commercial purposes), look for a link on the right column.
If you do not want to use DIPimage, you’ll have to add 1 to all indexing, and also switch the position of the first two indices (
a = a(1:imsize(a,1)-s(1))).Firstly I would like to thank you for you detailed explanations, but Cris, unfortunately when i am running the above script
I am getting the following erro
Error using dip_image/cat (line 100)
CAT arguments dimensions are not consistent.
Error in dip_image/iterate (line 89)
a(jj) = feval(fun,args{:});
Error in panor (line 92)
stitch = iterate(‘cat’,1,stitch2,stitch3,stitch4);
Could you please help me to debug this. Thanks
Hi prashanth,
I’m not sure why you get this error, the script works well for me. Make sure
imsize(stitch2)returns the same image size asimsize(stitch3)andimsize(stitch4). In any case, this line just puts 3 different outputs together to display them side-by-side on the web page; it is not a part of the stitching algorithm.Hi Cris,
% Alternative: the composition I made in the original blog post
out = readim(‘bigsur.jpg’);
stitch3 = out(sz(1)+s(1):sz(1)-1,:);
I get error “”bigsur.jpg” not found.”
and when I make this ‘bigsur1.jpg’ or ‘bigsur2.jpg’, I get the same error as prashanth got above,
??? Error using ==> dip_image.cat at 100
CAT arguments dimensions are not consistent.
Error in ==> dip_image.iterate at 89
a(jj) = feval(fun,args{:});
Error in ==> bigsur_new at 92
stitch = iterate(‘cat’,1,stitch2,stitch3,stitch4);
please help me to sort it out.
Hi Ramzan,
The file ‘bigsur.jpg’ is the stitched result from the first post on panoramic stitching that I wrote (linked in the beginning of this post). If you use that image, the dimensions for the three little images should match and the ‘cat’ command should work fine. But as I said to Prashanth above, this part of the script is only to make the image comparing the three stitching methods, it is not necessary at all for stitching.
Dear Cris, a bundle of thanks for helping me out. Everything is fine now. I am novice to Matlab and image stitching but I have learned a lot from your code. My sincere salute to you, People like you are great asset to the learner like me.