Superpixels are used in image segmentation as a pre-processing step. Instead of segmenting pixels directly, we first group similar pixels into “super-pixels”, which can then be processed further (and more cheaply).
(image from Wikimedia)
The current version of imager doesn’t implement them, but it turns out that SLIC superpixels are particularly easy to implement. SLIC is essentially k-means applied to pixels, with some bells and whistles.
We could use k-means to segment images based on colour alone. To get good results on colour segmentation the CIELAB colour space is appropriate, because it tries to be perceptually uniform.
library(tidyverse) library(imager) im <- load.image("https://upload.wikimedia.org/wikipedia/commons/thumb/f/fd/Aster_Tataricus.JPG/1024px-Aster_Tataricus.JPG") #Convert to CIELAB colour space, then create a data.frame with three colour channels as columns d <- sRGBtoLab(im) %>% as.data.frame(wide="c")%>% dplyr::select(-x,-y) #Run k-means with 2 centers km <- kmeans(d,2) #Turn cluster index into an image seg <- as.cimg(km$cluster,dim=c(dim(im)[1:2],1,1)) plot(im,axes=FALSE) highlight(seg==1)
We mostly manage to separate the petals from the rest, with a few errors here and there.
SLIC does pretty much the same thing, except we (a) use many more centers and (b) we add pixel coordinates as features in the clustering. The latter ensures that only adjacent pixels get grouped together.
The code below implements SLIC. It’s mostly straightforward:
#Compute SLIC superpixels #im: input image #nS: number of superpixels #ratio: determines compactness of superpixels. #low values will result in pixels with weird shapes #... further arguments passed to kmeans slic <- function(im,nS,compactness=1,...) { #If image is in colour, convert to CIELAB if (spectrum(im) ==3) im <- sRGBtoLab(im) #The pixel coordinates vary over 1...width(im) and 1...height(im) #Pixel values can be over a widely different range #We need our features to have similar scales, so #we compute relative scales of spatial dimensions to colour dimensions sc.spat <- (dim(im)[1:2]*.28) %>% max #Scale of spatial dimensions sc.col <- imsplit(im,"c") %>% map_dbl(sd) %>% max #Scaling ratio for pixel values rat <- (sc.spat/sc.col)/(compactness*10) X <- as.data.frame(im*rat,wide="c") %>% as.matrix #Generate initial centers from a grid ind <- round(seq(1,nPix(im)/spectrum(im),l=nS)) #Run k-means km <- kmeans(X,X[ind,],...) #Return segmentation as image (pixel values index cluster) seg <- as.cimg(km$cluster,dim=c(dim(im)[1:2],1,1)) #Superpixel image: each pixel is given the colour of the superpixel it belongs to sp <- map(1:spectrum(im),~ km$centers[km$cluster,2+.]) %>% do.call(c,.) %>% as.cimg(dim=dim(im)) #Correct for ratio sp <- sp/rat if (spectrum(im)==3) { #Convert back to RGB sp <- LabtosRGB(sp) } list(km=km,seg=seg,sp=sp) }
Use it as follows:
#400 superpixels out <- slic(im,400) #Superpixels plot(out$sp,axes=FALSE) #Segmentation plot(out$seg,axes=FALSE) #Show segmentation on original image (im*add.colour(abs(imlap(out$seg)) == 0)) %>% plot(axes=FALSE)
The next step is to segment the superpixels but I’ll keep that for another time.
Update: as reported on Github, some Windows users might run into a problem when loading the test image directly from a URL. The solution is to add Wikimedia to your list of trusted sites, see
I tried this. It was fun. I await the next installment
Hi, exists some way to get superpixel segmented?
I’m not sure what you mean?
hello, I tried this it works quiet well. I have used it on my image and its good. thanks for sharing. I tried exporting segments but it gives me an error message…see below
> writeOGR(seg, dsn = ‘.’, layer = ‘seg’, driver = ‘ESRI Shapefile’)
Error: inherits(obj, “Spatial”) is not TRUE
Please can any one help? I am a beginner here with r and image analysis.
Thanks
Looks like you’re trying to convert segments into polygons. I’ve no idea what sort of format writeOGR expects, and in any case it doesn’t sound like an easy task for a beginner. You might want to try the R-Sig-Geo list, but you should probably learn some more about spatial data in R first.
Hello Simon, I have explored and realised that the segments that result from SLIC, are of the class ‘Pixset’ “imager_array” “logical”. In this format, i have tried to give it spatial reference but it has not been possible. I tried to convert it to a polygon, could not. Please, I need help.
As a beginner, I am also exploring hard. help is highly appreciated.
How do I reference part of a code I got from online like the superpixel segmentation here?