可以使用哪些图像处理技术来实现检测以下图像中显示的圣诞树的应用程序?
我正在寻找解决方案，将工作在所有这些图像。因此，需要训练haar级联分类器或模板匹配的方法不是很有趣。
我正在寻找可以用任何编程语言编写的东西，只要它只使用开源技术。解决方案必须使用在此问题上共享的图像进行测试。有6个输入图像，答案应该显示处理每个图像的结果。最后，对于每个输出图像，必须在检测到的树周围绘制红线。
您将如何通过编程来检测这些图像中的树木呢?
当前回答
一些老式的图像处理方法…… 这个想法是基于这样的假设，即图像描绘的是在通常较暗和较光滑的背景(在某些情况下是前景)上点亮的树木。点亮的树木区域更“有活力”，具有更高的强度。 具体流程如下:
转换为灰度 应用LoG过滤来获得最“活跃”的区域 应用亮度阈值来获得最亮的区域 结合前两个得到一个初步的蒙版 应用形态扩张来扩大区域并连接相邻组件 根据候选区域的面积大小剔除较小的候选区域
你得到的是一个二进制掩码和每个图像的包围框。
以下是使用这种简单技术的结果:
MATLAB代码如下: 该代码运行在带有JPG图像的文件夹上。加载所有图像并返回检测到的结果。
% clear everything
clear;
pack;
close all;
close all hidden;
drawnow;
clc;
% initialization
ims=dir('./*.jpg');
imgs={};
images={};
blur_images={};
log_image={};
dilated_image={};
int_image={};
bin_image={};
measurements={};
box={};
num=length(ims);
thres_div = 3;
for i=1:num,
% load original image
imgs{end+1}=imread(ims(i).name);
% convert to grayscale
images{end+1}=rgb2gray(imgs{i});
% apply laplacian filtering and heuristic hard thresholding
val_thres = (max(max(images{i}))/thres_div);
log_image{end+1} = imfilter( images{i},fspecial('log')) > val_thres;
% get the most bright regions of the image
int_thres = 0.26*max(max( images{i}));
int_image{end+1} = images{i} > int_thres;
% compute the final binary image by combining
% high 'activity' with high intensity
bin_image{end+1} = log_image{i} .* int_image{i};
% apply morphological dilation to connect distonnected components
strel_size = round(0.01*max(size(imgs{i}))); % structuring element for morphological dilation
dilated_image{end+1} = imdilate( bin_image{i}, strel('disk',strel_size));
% do some measurements to eliminate small objects
measurements{i} = regionprops( logical( dilated_image{i}),'Area','BoundingBox');
for m=1:length(measurements{i})
if measurements{i}(m).Area < 0.05*numel( dilated_image{i})
dilated_image{i}( round(measurements{i}(m).BoundingBox(2):measurements{i}(m).BoundingBox(4)+measurements{i}(m).BoundingBox(2)),...
round(measurements{i}(m).BoundingBox(1):measurements{i}(m).BoundingBox(3)+measurements{i}(m).BoundingBox(1))) = 0;
end
end
% make sure the dilated image is the same size with the original
dilated_image{i} = dilated_image{i}(1:size(imgs{i},1),1:size(imgs{i},2));
% compute the bounding box
[y,x] = find( dilated_image{i});
if isempty( y)
box{end+1}=[];
else
box{end+1} = [ min(x) min(y) max(x)-min(x)+1 max(y)-min(y)+1];
end
end
%%% additional code to display things
for i=1:num,
figure;
subplot(121);
colormap gray;
imshow( imgs{i});
if ~isempty(box{i})
hold on;
rr = rectangle( 'position', box{i});
set( rr, 'EdgeColor', 'r');
hold off;
end
subplot(122);
imshow( imgs{i}.*uint8(repmat(dilated_image{i},[1 1 3])));
end
其他回答
代码用Matlab R2007a编写。我用k-means粗略地提取了圣诞树。我 将只显示一张图像的中间结果，并显示所有六张图像的最终结果。
首先，我将RGB空间映射到Lab空间上，这样可以增强红色b通道的对比度:
colorTransform = makecform('srgb2lab');
I = applycform(I, colorTransform);
L = double(I(:,:,1));
a = double(I(:,:,2));
b = double(I(:,:,3));
除了颜色空间的特征，我还使用了与纹理相关的特征 而不是每个像素本身。这里我线性组合了强度 3个原始频道(R、G、B)。我之所以这样格式化是因为圣诞节 图中的树都有红灯，有时是绿色，有时是蓝色 还有照明。
R=double(Irgb(:,:,1));
G=double(Irgb(:,:,2));
B=double(Irgb(:,:,3));
I0 = (3*R + max(G,B)-min(G,B))/2;
我在I0上应用了一个3X3局部二进制模式，以中心像素作为阈值，并且 通过计算平均像素强度值的差值得到对比度 高于阈值，平均值低于阈值。
I0_copy = zeros(size(I0));
for i = 2 : size(I0,1) - 1
for j = 2 : size(I0,2) - 1
tmp = I0(i-1:i+1,j-1:j+1) >= I0(i,j);
I0_copy(i,j) = mean(mean(tmp.*I0(i-1:i+1,j-1:j+1))) - ...
mean(mean(~tmp.*I0(i-1:i+1,j-1:j+1))); % Contrast
end
end
因为我总共有4个特征，所以在我的聚类方法中我会选择K=5。的代码 k-均值如下所示(它来自Andrew Ng博士的机器学习课程。我拿了 在他的编程作业中，我自己写了代码)。
[centroids, idx] = runkMeans(X, initial_centroids, max_iters);
mask=reshape(idx,img_size(1),img_size(2));
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function [centroids, idx] = runkMeans(X, initial_centroids, ...
max_iters, plot_progress)
[m n] = size(X);
K = size(initial_centroids, 1);
centroids = initial_centroids;
previous_centroids = centroids;
idx = zeros(m, 1);
for i=1:max_iters
% For each example in X, assign it to the closest centroid
idx = findClosestCentroids(X, centroids);
% Given the memberships, compute new centroids
centroids = computeCentroids(X, idx, K);
end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function idx = findClosestCentroids(X, centroids)
K = size(centroids, 1);
idx = zeros(size(X,1), 1);
for xi = 1:size(X,1)
x = X(xi, :);
% Find closest centroid for x.
best = Inf;
for mui = 1:K
mu = centroids(mui, :);
d = dot(x - mu, x - mu);
if d < best
best = d;
idx(xi) = mui;
end
end
end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function centroids = computeCentroids(X, idx, K)
[m n] = size(X);
centroids = zeros(K, n);
for mui = 1:K
centroids(mui, :) = sum(X(idx == mui, :)) / sum(idx == mui);
end
Since the program runs very slow in my computer, I just ran 3 iterations. Normally the stop criteria is (i) iteration time at least 10, or (ii) no change on the centroids any more. To my test, increasing the iteration may differentiate the background (sky and tree, sky and building,...) more accurately, but did not show a drastic changes in christmas tree extraction. Also note k-means is not immune to the random centroid initialization, so running the program several times to make a comparison is recommended.
After the k-means, the labelled region with the maximum intensity of I0 was chosen. And boundary tracing was used to extracted the boundaries. To me, the last christmas tree is the most difficult one to extract since the contrast in that picture is not high enough as they are in the first five. Another issue in my method is that I used bwboundaries function in Matlab to trace the boundary, but sometimes the inner boundaries are also included as you can observe in 3rd, 5th, 6th results. The dark side within the christmas trees are not only failed to be clustered with the illuminated side, but they also lead to so many tiny inner boundaries tracing (imfill doesn't improve very much). In all my algorithm still has a lot improvement space.
Some publications indicates that mean-shift may be more robust than k-means, and many graph-cut based algorithms are also very competitive on complicated boundaries segmentation. I wrote a mean-shift algorithm myself, it seems to better extract the regions without enough light. But mean-shift is a little bit over-segmented, and some strategy of merging is needed. It ran even much slower than k-means in my computer, I am afraid I have to give it up. I eagerly look forward to see others would submit excellent results here with those modern algorithms mentioned above.
但我始终认为特征选择是图像分割的关键部分。与 一个适当的特征选择，可以最大化的边缘之间的对象和背景，许多 分割算法肯定会起作用。不同的算法可以改善结果 从1到10，但特征选择可以将其从0提高到1。
圣诞快乐!
一些老式的图像处理方法…… 这个想法是基于这样的假设，即图像描绘的是在通常较暗和较光滑的背景(在某些情况下是前景)上点亮的树木。点亮的树木区域更“有活力”，具有更高的强度。 具体流程如下:
转换为灰度 应用LoG过滤来获得最“活跃”的区域 应用亮度阈值来获得最亮的区域 结合前两个得到一个初步的蒙版 应用形态扩张来扩大区域并连接相邻组件 根据候选区域的面积大小剔除较小的候选区域
你得到的是一个二进制掩码和每个图像的包围框。
以下是使用这种简单技术的结果:
MATLAB代码如下: 该代码运行在带有JPG图像的文件夹上。加载所有图像并返回检测到的结果。
% clear everything
clear;
pack;
close all;
close all hidden;
drawnow;
clc;
% initialization
ims=dir('./*.jpg');
imgs={};
images={};
blur_images={};
log_image={};
dilated_image={};
int_image={};
bin_image={};
measurements={};
box={};
num=length(ims);
thres_div = 3;
for i=1:num,
% load original image
imgs{end+1}=imread(ims(i).name);
% convert to grayscale
images{end+1}=rgb2gray(imgs{i});
% apply laplacian filtering and heuristic hard thresholding
val_thres = (max(max(images{i}))/thres_div);
log_image{end+1} = imfilter( images{i},fspecial('log')) > val_thres;
% get the most bright regions of the image
int_thres = 0.26*max(max( images{i}));
int_image{end+1} = images{i} > int_thres;
% compute the final binary image by combining
% high 'activity' with high intensity
bin_image{end+1} = log_image{i} .* int_image{i};
% apply morphological dilation to connect distonnected components
strel_size = round(0.01*max(size(imgs{i}))); % structuring element for morphological dilation
dilated_image{end+1} = imdilate( bin_image{i}, strel('disk',strel_size));
% do some measurements to eliminate small objects
measurements{i} = regionprops( logical( dilated_image{i}),'Area','BoundingBox');
for m=1:length(measurements{i})
if measurements{i}(m).Area < 0.05*numel( dilated_image{i})
dilated_image{i}( round(measurements{i}(m).BoundingBox(2):measurements{i}(m).BoundingBox(4)+measurements{i}(m).BoundingBox(2)),...
round(measurements{i}(m).BoundingBox(1):measurements{i}(m).BoundingBox(3)+measurements{i}(m).BoundingBox(1))) = 0;
end
end
% make sure the dilated image is the same size with the original
dilated_image{i} = dilated_image{i}(1:size(imgs{i},1),1:size(imgs{i},2));
% compute the bounding box
[y,x] = find( dilated_image{i});
if isempty( y)
box{end+1}=[];
else
box{end+1} = [ min(x) min(y) max(x)-min(x)+1 max(y)-min(y)+1];
end
end
%%% additional code to display things
for i=1:num,
figure;
subplot(121);
colormap gray;
imshow( imgs{i});
if ~isempty(box{i})
hold on;
rr = rectangle( 'position', box{i});
set( rr, 'EdgeColor', 'r');
hold off;
end
subplot(122);
imshow( imgs{i}.*uint8(repmat(dilated_image{i},[1 1 3])));
end
我有一个方法，我认为是有趣的，有点不同于其他。与其他一些方法相比，我的方法的主要区别在于如何执行图像分割步骤——我使用了Python的scikit-learn中的DBSCAN聚类算法;它被优化用于寻找一些无定形的形状，这些形状可能不一定有一个清晰的形心。
At the top level, my approach is fairly simple and can be broken down into about 3 steps. First I apply a threshold (or actually, the logical "or" of two separate and distinct thresholds). As with many of the other answers, I assumed that the Christmas tree would be one of the brighter objects in the scene, so the first threshold is just a simple monochrome brightness test; any pixels with values above 220 on a 0-255 scale (where black is 0 and white is 255) are saved to a binary black-and-white image. The second threshold tries to look for red and yellow lights, which are particularly prominent in the trees in the upper left and lower right of the six images, and stand out well against the blue-green background which is prevalent in most of the photos. I convert the rgb image to hsv space, and require that the hue is either less than 0.2 on a 0.0-1.0 scale (corresponding roughly to the border between yellow and green) or greater than 0.95 (corresponding to the border between purple and red) and additionally I require bright, saturated colors: saturation and value must both be above 0.7. The results of the two threshold procedures are logically "or"-ed together, and the resulting matrix of black-and-white binary images is shown below:
您可以清楚地看到，每张图像都有一个大的像素集群，大致对应于每棵树的位置，加上一些图像还有一些其他的小集群，对应于一些建筑物窗户上的灯光，或地平线上的背景场景。下一步是让计算机识别出这些是独立的集群，并用集群成员ID号正确地标记每个像素。
对于这个任务，我选择了DBSCAN。这里有一个关于DBSCAN与其他聚类算法的典型行为的很好的可视化比较。正如我前面所说，它可以很好地处理非晶态形状。DBSCAN的输出，每个集群用不同的颜色绘制，如下所示:
There are a few things to be aware of when looking at this result. First is that DBSCAN requires the user to set a "proximity" parameter in order to regulate its behavior, which effectively controls how separated a pair of points must be in order for the algorithm to declare a new separate cluster rather than agglomerating a test point onto an already pre-existing cluster. I set this value to be 0.04 times the size along the diagonal of each image. Since the images vary in size from roughly VGA up to about HD 1080, this type of scale-relative definition is critical.
Another point worth noting is that the DBSCAN algorithm as it is implemented in scikit-learn has memory limits which are fairly challenging for some of the larger images in this sample. Therefore, for a few of the larger images, I actually had to "decimate" (i.e., retain only every 3rd or 4th pixel and drop the others) each cluster in order to stay within this limit. As a result of this culling process, the remaining individual sparse pixels are difficult to see on some of the larger images. Therefore, for display purposes only, the color-coded pixels in the above images have been effectively "dilated" just slightly so that they stand out better. It's purely a cosmetic operation for the sake of the narrative; although there are comments mentioning this dilation in my code, rest assured that it has nothing to do with any calculations that actually matter.
一旦识别并标记了集群，第三步也是最后一步就很容易了:我只需要在每个图像中选择最大的集群(在这种情况下，我选择根据成员像素的总数来衡量“大小”，尽管也可以使用某种类型的度量标准来衡量物理范围)，然后计算该集群的凸包。凸包就变成了树的边界。通过该方法计算的6个凸包如下图中红色部分所示:
源代码是为Python 2.7.6编写的，它依赖于numpy, scipy, matplotlib和scikit-learn。我把它分成了两部分。第一部分负责实际图像处理:
from PIL import Image
import numpy as np
import scipy as sp
import matplotlib.colors as colors
from sklearn.cluster import DBSCAN
from math import ceil, sqrt
"""
Inputs:
rgbimg: [M,N,3] numpy array containing (uint, 0-255) color image
hueleftthr: Scalar constant to select maximum allowed hue in the
yellow-green region
huerightthr: Scalar constant to select minimum allowed hue in the
blue-purple region
satthr: Scalar constant to select minimum allowed saturation
valthr: Scalar constant to select minimum allowed value
monothr: Scalar constant to select minimum allowed monochrome
brightness
maxpoints: Scalar constant maximum number of pixels to forward to
the DBSCAN clustering algorithm
proxthresh: Proximity threshold to use for DBSCAN, as a fraction of
the diagonal size of the image
Outputs:
borderseg: [K,2,2] Nested list containing K pairs of x- and y- pixel
values for drawing the tree border
X: [P,2] List of pixels that passed the threshold step
labels: [Q,2] List of cluster labels for points in Xslice (see
below)
Xslice: [Q,2] Reduced list of pixels to be passed to DBSCAN
"""
def findtree(rgbimg, hueleftthr=0.2, huerightthr=0.95, satthr=0.7,
valthr=0.7, monothr=220, maxpoints=5000, proxthresh=0.04):
# Convert rgb image to monochrome for
gryimg = np.asarray(Image.fromarray(rgbimg).convert('L'))
# Convert rgb image (uint, 0-255) to hsv (float, 0.0-1.0)
hsvimg = colors.rgb_to_hsv(rgbimg.astype(float)/255)
# Initialize binary thresholded image
binimg = np.zeros((rgbimg.shape[0], rgbimg.shape[1]))
# Find pixels with hue<0.2 or hue>0.95 (red or yellow) and saturation/value
# both greater than 0.7 (saturated and bright)--tends to coincide with
# ornamental lights on trees in some of the images
boolidx = np.logical_and(
np.logical_and(
np.logical_or((hsvimg[:,:,0] < hueleftthr),
(hsvimg[:,:,0] > huerightthr)),
(hsvimg[:,:,1] > satthr)),
(hsvimg[:,:,2] > valthr))
# Find pixels that meet hsv criterion
binimg[np.where(boolidx)] = 255
# Add pixels that meet grayscale brightness criterion
binimg[np.where(gryimg > monothr)] = 255
# Prepare thresholded points for DBSCAN clustering algorithm
X = np.transpose(np.where(binimg == 255))
Xslice = X
nsample = len(Xslice)
if nsample > maxpoints:
# Make sure number of points does not exceed DBSCAN maximum capacity
Xslice = X[range(0,nsample,int(ceil(float(nsample)/maxpoints)))]
# Translate DBSCAN proximity threshold to units of pixels and run DBSCAN
pixproxthr = proxthresh * sqrt(binimg.shape[0]**2 + binimg.shape[1]**2)
db = DBSCAN(eps=pixproxthr, min_samples=10).fit(Xslice)
labels = db.labels_.astype(int)
# Find the largest cluster (i.e., with most points) and obtain convex hull
unique_labels = set(labels)
maxclustpt = 0
for k in unique_labels:
class_members = [index[0] for index in np.argwhere(labels == k)]
if len(class_members) > maxclustpt:
points = Xslice[class_members]
hull = sp.spatial.ConvexHull(points)
maxclustpt = len(class_members)
borderseg = [[points[simplex,0], points[simplex,1]] for simplex
in hull.simplices]
return borderseg, X, labels, Xslice
第二部分是一个用户级脚本，它调用第一个文件并生成上面所有的图:
#!/usr/bin/env python
from PIL import Image
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.cm as cm
from findtree import findtree
# Image files to process
fname = ['nmzwj.png', 'aVZhC.png', '2K9EF.png',
'YowlH.png', '2y4o5.png', 'FWhSP.png']
# Initialize figures
fgsz = (16,7)
figthresh = plt.figure(figsize=fgsz, facecolor='w')
figclust = plt.figure(figsize=fgsz, facecolor='w')
figcltwo = plt.figure(figsize=fgsz, facecolor='w')
figborder = plt.figure(figsize=fgsz, facecolor='w')
figthresh.canvas.set_window_title('Thresholded HSV and Monochrome Brightness')
figclust.canvas.set_window_title('DBSCAN Clusters (Raw Pixel Output)')
figcltwo.canvas.set_window_title('DBSCAN Clusters (Slightly Dilated for Display)')
figborder.canvas.set_window_title('Trees with Borders')
for ii, name in zip(range(len(fname)), fname):
# Open the file and convert to rgb image
rgbimg = np.asarray(Image.open(name))
# Get the tree borders as well as a bunch of other intermediate values
# that will be used to illustrate how the algorithm works
borderseg, X, labels, Xslice = findtree(rgbimg)
# Display thresholded images
axthresh = figthresh.add_subplot(2,3,ii+1)
axthresh.set_xticks([])
axthresh.set_yticks([])
binimg = np.zeros((rgbimg.shape[0], rgbimg.shape[1]))
for v, h in X:
binimg[v,h] = 255
axthresh.imshow(binimg, interpolation='nearest', cmap='Greys')
# Display color-coded clusters
axclust = figclust.add_subplot(2,3,ii+1) # Raw version
axclust.set_xticks([])
axclust.set_yticks([])
axcltwo = figcltwo.add_subplot(2,3,ii+1) # Dilated slightly for display only
axcltwo.set_xticks([])
axcltwo.set_yticks([])
axcltwo.imshow(binimg, interpolation='nearest', cmap='Greys')
clustimg = np.ones(rgbimg.shape)
unique_labels = set(labels)
# Generate a unique color for each cluster
plcol = cm.rainbow_r(np.linspace(0, 1, len(unique_labels)))
for lbl, pix in zip(labels, Xslice):
for col, unqlbl in zip(plcol, unique_labels):
if lbl == unqlbl:
# Cluster label of -1 indicates no cluster membership;
# override default color with black
if lbl == -1:
col = [0.0, 0.0, 0.0, 1.0]
# Raw version
for ij in range(3):
clustimg[pix[0],pix[1],ij] = col[ij]
# Dilated just for display
axcltwo.plot(pix[1], pix[0], 'o', markerfacecolor=col,
markersize=1, markeredgecolor=col)
axclust.imshow(clustimg)
axcltwo.set_xlim(0, binimg.shape[1]-1)
axcltwo.set_ylim(binimg.shape[0], -1)
# Plot original images with read borders around the trees
axborder = figborder.add_subplot(2,3,ii+1)
axborder.set_axis_off()
axborder.imshow(rgbimg, interpolation='nearest')
for vseg, hseg in borderseg:
axborder.plot(hseg, vseg, 'r-', lw=3)
axborder.set_xlim(0, binimg.shape[1]-1)
axborder.set_ylim(binimg.shape[0], -1)
plt.show()
这是我简单而愚蠢的解决方案。 它是基于这样一个假设:树将是图片中最明亮、最大的东西。
//g++ -Wall -pedantic -ansi -O2 -pipe -s -o christmas_tree christmas_tree.cpp `pkg-config --cflags --libs opencv`
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <iostream>
using namespace cv;
using namespace std;
int main(int argc,char *argv[])
{
Mat original,tmp,tmp1;
vector <vector<Point> > contours;
Moments m;
Rect boundrect;
Point2f center;
double radius, max_area=0,tmp_area=0;
unsigned int j, k;
int i;
for(i = 1; i < argc; ++i)
{
original = imread(argv[i]);
if(original.empty())
{
cerr << "Error"<<endl;
return -1;
}
GaussianBlur(original, tmp, Size(3, 3), 0, 0, BORDER_DEFAULT);
erode(tmp, tmp, Mat(), Point(-1, -1), 10);
cvtColor(tmp, tmp, CV_BGR2HSV);
inRange(tmp, Scalar(0, 0, 0), Scalar(180, 255, 200), tmp);
dilate(original, tmp1, Mat(), Point(-1, -1), 15);
cvtColor(tmp1, tmp1, CV_BGR2HLS);
inRange(tmp1, Scalar(0, 185, 0), Scalar(180, 255, 255), tmp1);
dilate(tmp1, tmp1, Mat(), Point(-1, -1), 10);
bitwise_and(tmp, tmp1, tmp1);
findContours(tmp1, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE);
max_area = 0;
j = 0;
for(k = 0; k < contours.size(); k++)
{
tmp_area = contourArea(contours[k]);
if(tmp_area > max_area)
{
max_area = tmp_area;
j = k;
}
}
tmp1 = Mat::zeros(original.size(),CV_8U);
approxPolyDP(contours[j], contours[j], 30, true);
drawContours(tmp1, contours, j, Scalar(255,255,255), CV_FILLED);
m = moments(contours[j]);
boundrect = boundingRect(contours[j]);
center = Point2f(m.m10/m.m00, m.m01/m.m00);
radius = (center.y - (boundrect.tl().y))/4.0*3.0;
Rect heightrect(center.x-original.cols/5, boundrect.tl().y, original.cols/5*2, boundrect.size().height);
tmp = Mat::zeros(original.size(), CV_8U);
rectangle(tmp, heightrect, Scalar(255, 255, 255), -1);
circle(tmp, center, radius, Scalar(255, 255, 255), -1);
bitwise_and(tmp, tmp1, tmp1);
findContours(tmp1, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE);
max_area = 0;
j = 0;
for(k = 0; k < contours.size(); k++)
{
tmp_area = contourArea(contours[k]);
if(tmp_area > max_area)
{
max_area = tmp_area;
j = k;
}
}
approxPolyDP(contours[j], contours[j], 30, true);
convexHull(contours[j], contours[j]);
drawContours(original, contours, j, Scalar(0, 0, 255), 3);
namedWindow(argv[i], CV_WINDOW_NORMAL|CV_WINDOW_KEEPRATIO|CV_GUI_EXPANDED);
imshow(argv[i], original);
waitKey(0);
destroyWindow(argv[i]);
}
return 0;
}
第一步是检测图片中最亮的像素，但我们必须在树本身和反射其光的雪之间做区分。在这里，我们试图排除雪应用一个非常简单的滤镜的颜色代码:
GaussianBlur(original, tmp, Size(3, 3), 0, 0, BORDER_DEFAULT);
erode(tmp, tmp, Mat(), Point(-1, -1), 10);
cvtColor(tmp, tmp, CV_BGR2HSV);
inRange(tmp, Scalar(0, 0, 0), Scalar(180, 255, 200), tmp);
然后我们找到每个“亮”像素:
dilate(original, tmp1, Mat(), Point(-1, -1), 15);
cvtColor(tmp1, tmp1, CV_BGR2HLS);
inRange(tmp1, Scalar(0, 185, 0), Scalar(180, 255, 255), tmp1);
dilate(tmp1, tmp1, Mat(), Point(-1, -1), 10);
最后我们将两个结果结合起来:
bitwise_and(tmp, tmp1, tmp1);
现在我们寻找最大的明亮物体:
findContours(tmp1, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE);
max_area = 0;
j = 0;
for(k = 0; k < contours.size(); k++)
{
tmp_area = contourArea(contours[k]);
if(tmp_area > max_area)
{
max_area = tmp_area;
j = k;
}
}
tmp1 = Mat::zeros(original.size(),CV_8U);
approxPolyDP(contours[j], contours[j], 30, true);
drawContours(tmp1, contours, j, Scalar(255,255,255), CV_FILLED);
现在我们已经基本完成了，但是由于下雪，我们仍然有一些不完善的地方。 为了切断它们，我们将使用一个圆形和一个矩形来近似树的形状来创建一个蒙版，以删除不需要的部分:
m = moments(contours[j]);
boundrect = boundingRect(contours[j]);
center = Point2f(m.m10/m.m00, m.m01/m.m00);
radius = (center.y - (boundrect.tl().y))/4.0*3.0;
Rect heightrect(center.x-original.cols/5, boundrect.tl().y, original.cols/5*2, boundrect.size().height);
tmp = Mat::zeros(original.size(), CV_8U);
rectangle(tmp, heightrect, Scalar(255, 255, 255), -1);
circle(tmp, center, radius, Scalar(255, 255, 255), -1);
bitwise_and(tmp, tmp1, tmp1);
最后一步是找到我们的树的轮廓，并把它画在原图上。
findContours(tmp1, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE);
max_area = 0;
j = 0;
for(k = 0; k < contours.size(); k++)
{
tmp_area = contourArea(contours[k]);
if(tmp_area > max_area)
{
max_area = tmp_area;
j = k;
}
}
approxPolyDP(contours[j], contours[j], 30, true);
convexHull(contours[j], contours[j]);
drawContours(original, contours, j, Scalar(0, 0, 255), 3);
很抱歉，目前我的网络连接不好，无法上传图片。我以后再试着做。
圣诞快乐。
编辑:
以下是最终输出的一些图片:
编辑注意:我编辑这篇文章是为了(I)按照要求分别处理每个树图像，(ii)同时考虑物体亮度和形状，以提高结果的质量。
下面是一种考虑到物体亮度和形状的方法。换句话说，它寻找的是具有三角形形状和显著亮度的物体。它是用Java语言实现的，使用Marvin图像处理框架。
第一步是颜色阈值。这里的目标是集中分析具有显著亮度的物体。
输出图片:
源代码:
public class ChristmasTree {
private MarvinImagePlugin fill = MarvinPluginLoader.loadImagePlugin("org.marvinproject.image.fill.boundaryFill");
private MarvinImagePlugin threshold = MarvinPluginLoader.loadImagePlugin("org.marvinproject.image.color.thresholding");
private MarvinImagePlugin invert = MarvinPluginLoader.loadImagePlugin("org.marvinproject.image.color.invert");
private MarvinImagePlugin dilation = MarvinPluginLoader.loadImagePlugin("org.marvinproject.image.morphological.dilation");
public ChristmasTree(){
MarvinImage tree;
// Iterate each image
for(int i=1; i<=6; i++){
tree = MarvinImageIO.loadImage("./res/trees/tree"+i+".png");
// 1. Threshold
threshold.setAttribute("threshold", 200);
threshold.process(tree.clone(), tree);
}
}
public static void main(String[] args) {
new ChristmasTree();
}
}
在第二步中，图像中最亮的点被放大以形成形状。这一过程的结果是具有显著亮度的物体的可能形状。应用洪水填充分割，断开的形状被检测。
输出图片:
源代码:
public class ChristmasTree {
private MarvinImagePlugin fill = MarvinPluginLoader.loadImagePlugin("org.marvinproject.image.fill.boundaryFill");
private MarvinImagePlugin threshold = MarvinPluginLoader.loadImagePlugin("org.marvinproject.image.color.thresholding");
private MarvinImagePlugin invert = MarvinPluginLoader.loadImagePlugin("org.marvinproject.image.color.invert");
private MarvinImagePlugin dilation = MarvinPluginLoader.loadImagePlugin("org.marvinproject.image.morphological.dilation");
public ChristmasTree(){
MarvinImage tree;
// Iterate each image
for(int i=1; i<=6; i++){
tree = MarvinImageIO.loadImage("./res/trees/tree"+i+".png");
// 1. Threshold
threshold.setAttribute("threshold", 200);
threshold.process(tree.clone(), tree);
// 2. Dilate
invert.process(tree.clone(), tree);
tree = MarvinColorModelConverter.rgbToBinary(tree, 127);
MarvinImageIO.saveImage(tree, "./res/trees/new/tree_"+i+"threshold.png");
dilation.setAttribute("matrix", MarvinMath.getTrueMatrix(50, 50));
dilation.process(tree.clone(), tree);
MarvinImageIO.saveImage(tree, "./res/trees/new/tree_"+1+"_dilation.png");
tree = MarvinColorModelConverter.binaryToRgb(tree);
// 3. Segment shapes
MarvinImage trees2 = tree.clone();
fill(tree, trees2);
MarvinImageIO.saveImage(trees2, "./res/trees/new/tree_"+i+"_fill.png");
}
private void fill(MarvinImage imageIn, MarvinImage imageOut){
boolean found;
int color= 0xFFFF0000;
while(true){
found=false;
Outerloop:
for(int y=0; y<imageIn.getHeight(); y++){
for(int x=0; x<imageIn.getWidth(); x++){
if(imageOut.getIntComponent0(x, y) == 0){
fill.setAttribute("x", x);
fill.setAttribute("y", y);
fill.setAttribute("color", color);
fill.setAttribute("threshold", 120);
fill.process(imageIn, imageOut);
color = newColor(color);
found = true;
break Outerloop;
}
}
}
if(!found){
break;
}
}
}
private int newColor(int color){
int red = (color & 0x00FF0000) >> 16;
int green = (color & 0x0000FF00) >> 8;
int blue = (color & 0x000000FF);
if(red <= green && red <= blue){
red+=5;
}
else if(green <= red && green <= blue){
green+=5;
}
else{
blue+=5;
}
return 0xFF000000 + (red << 16) + (green << 8) + blue;
}
public static void main(String[] args) {
new ChristmasTree();
}
}
如输出图像所示，检测到多个形状。在这个问题中，图像中只有几个亮点。但是，实现这种方法是为了处理更复杂的场景。
In the next step each shape is analyzed. A simple algorithm detects shapes with a pattern similar to a triangle. The algorithm analyze the object shape line by line. If the center of the mass of each shape line is almost the same (given a threshold) and mass increase as y increase, the object has a triangle-like shape. The mass of the shape line is the number of pixels in that line that belongs to the shape. Imagine you slice the object horizontally and analyze each horizontal segment. If they are centralized to each other and the length increase from the first segment to last one in a linear pattern, you probably has an object that resembles a triangle.
源代码:
private int[] detectTrees(MarvinImage image){
HashSet<Integer> analysed = new HashSet<Integer>();
boolean found;
while(true){
found = false;
for(int y=0; y<image.getHeight(); y++){
for(int x=0; x<image.getWidth(); x++){
int color = image.getIntColor(x, y);
if(!analysed.contains(color)){
if(isTree(image, color)){
return getObjectRect(image, color);
}
analysed.add(color);
found=true;
}
}
}
if(!found){
break;
}
}
return null;
}
private boolean isTree(MarvinImage image, int color){
int mass[][] = new int[image.getHeight()][2];
int yStart=-1;
int xStart=-1;
for(int y=0; y<image.getHeight(); y++){
int mc = 0;
int xs=-1;
int xe=-1;
for(int x=0; x<image.getWidth(); x++){
if(image.getIntColor(x, y) == color){
mc++;
if(yStart == -1){
yStart=y;
xStart=x;
}
if(xs == -1){
xs = x;
}
if(x > xe){
xe = x;
}
}
}
mass[y][0] = xs;
mass[y][3] = xe;
mass[y][4] = mc;
}
int validLines=0;
for(int y=0; y<image.getHeight(); y++){
if
(
mass[y][5] > 0 &&
Math.abs(((mass[y][0]+mass[y][6])/2)-xStart) <= 50 &&
mass[y][7] >= (mass[yStart][8] + (y-yStart)*0.3) &&
mass[y][9] <= (mass[yStart][10] + (y-yStart)*1.5)
)
{
validLines++;
}
}
if(validLines > 100){
return true;
}
return false;
}
最后，每个形状的位置类似于一个三角形，并具有显著的亮度，在这种情况下，圣诞树，突出显示在原图中，如下所示。
最终输出图片:
最终源代码:
public class ChristmasTree {
private MarvinImagePlugin fill = MarvinPluginLoader.loadImagePlugin("org.marvinproject.image.fill.boundaryFill");
private MarvinImagePlugin threshold = MarvinPluginLoader.loadImagePlugin("org.marvinproject.image.color.thresholding");
private MarvinImagePlugin invert = MarvinPluginLoader.loadImagePlugin("org.marvinproject.image.color.invert");
private MarvinImagePlugin dilation = MarvinPluginLoader.loadImagePlugin("org.marvinproject.image.morphological.dilation");
public ChristmasTree(){
MarvinImage tree;
// Iterate each image
for(int i=1; i<=6; i++){
tree = MarvinImageIO.loadImage("./res/trees/tree"+i+".png");
// 1. Threshold
threshold.setAttribute("threshold", 200);
threshold.process(tree.clone(), tree);
// 2. Dilate
invert.process(tree.clone(), tree);
tree = MarvinColorModelConverter.rgbToBinary(tree, 127);
MarvinImageIO.saveImage(tree, "./res/trees/new/tree_"+i+"threshold.png");
dilation.setAttribute("matrix", MarvinMath.getTrueMatrix(50, 50));
dilation.process(tree.clone(), tree);
MarvinImageIO.saveImage(tree, "./res/trees/new/tree_"+1+"_dilation.png");
tree = MarvinColorModelConverter.binaryToRgb(tree);
// 3. Segment shapes
MarvinImage trees2 = tree.clone();
fill(tree, trees2);
MarvinImageIO.saveImage(trees2, "./res/trees/new/tree_"+i+"_fill.png");
// 4. Detect tree-like shapes
int[] rect = detectTrees(trees2);
// 5. Draw the result
MarvinImage original = MarvinImageIO.loadImage("./res/trees/tree"+i+".png");
drawBoundary(trees2, original, rect);
MarvinImageIO.saveImage(original, "./res/trees/new/tree_"+i+"_out_2.jpg");
}
}
private void drawBoundary(MarvinImage shape, MarvinImage original, int[] rect){
int yLines[] = new int[6];
yLines[0] = rect[1];
yLines[1] = rect[1]+(int)((rect[3]/5));
yLines[2] = rect[1]+((rect[3]/5)*2);
yLines[3] = rect[1]+((rect[3]/5)*3);
yLines[4] = rect[1]+(int)((rect[3]/5)*4);
yLines[5] = rect[1]+rect[3];
List<Point> points = new ArrayList<Point>();
for(int i=0; i<yLines.length; i++){
boolean in=false;
Point startPoint=null;
Point endPoint=null;
for(int x=rect[0]; x<rect[0]+rect[2]; x++){
if(shape.getIntColor(x, yLines[i]) != 0xFFFFFFFF){
if(!in){
if(startPoint == null){
startPoint = new Point(x, yLines[i]);
}
}
in = true;
}
else{
if(in){
endPoint = new Point(x, yLines[i]);
}
in = false;
}
}
if(endPoint == null){
endPoint = new Point((rect[0]+rect[2])-1, yLines[i]);
}
points.add(startPoint);
points.add(endPoint);
}
drawLine(points.get(0).x, points.get(0).y, points.get(1).x, points.get(1).y, 15, original);
drawLine(points.get(1).x, points.get(1).y, points.get(3).x, points.get(3).y, 15, original);
drawLine(points.get(3).x, points.get(3).y, points.get(5).x, points.get(5).y, 15, original);
drawLine(points.get(5).x, points.get(5).y, points.get(7).x, points.get(7).y, 15, original);
drawLine(points.get(7).x, points.get(7).y, points.get(9).x, points.get(9).y, 15, original);
drawLine(points.get(9).x, points.get(9).y, points.get(11).x, points.get(11).y, 15, original);
drawLine(points.get(11).x, points.get(11).y, points.get(10).x, points.get(10).y, 15, original);
drawLine(points.get(10).x, points.get(10).y, points.get(8).x, points.get(8).y, 15, original);
drawLine(points.get(8).x, points.get(8).y, points.get(6).x, points.get(6).y, 15, original);
drawLine(points.get(6).x, points.get(6).y, points.get(4).x, points.get(4).y, 15, original);
drawLine(points.get(4).x, points.get(4).y, points.get(2).x, points.get(2).y, 15, original);
drawLine(points.get(2).x, points.get(2).y, points.get(0).x, points.get(0).y, 15, original);
}
private void drawLine(int x1, int y1, int x2, int y2, int length, MarvinImage image){
int lx1, lx2, ly1, ly2;
for(int i=0; i<length; i++){
lx1 = (x1+i >= image.getWidth() ? (image.getWidth()-1)-i: x1);
lx2 = (x2+i >= image.getWidth() ? (image.getWidth()-1)-i: x2);
ly1 = (y1+i >= image.getHeight() ? (image.getHeight()-1)-i: y1);
ly2 = (y2+i >= image.getHeight() ? (image.getHeight()-1)-i: y2);
image.drawLine(lx1+i, ly1, lx2+i, ly2, Color.red);
image.drawLine(lx1, ly1+i, lx2, ly2+i, Color.red);
}
}
private void fillRect(MarvinImage image, int[] rect, int length){
for(int i=0; i<length; i++){
image.drawRect(rect[0]+i, rect[1]+i, rect[2]-(i*2), rect[3]-(i*2), Color.red);
}
}
private void fill(MarvinImage imageIn, MarvinImage imageOut){
boolean found;
int color= 0xFFFF0000;
while(true){
found=false;
Outerloop:
for(int y=0; y<imageIn.getHeight(); y++){
for(int x=0; x<imageIn.getWidth(); x++){
if(imageOut.getIntComponent0(x, y) == 0){
fill.setAttribute("x", x);
fill.setAttribute("y", y);
fill.setAttribute("color", color);
fill.setAttribute("threshold", 120);
fill.process(imageIn, imageOut);
color = newColor(color);
found = true;
break Outerloop;
}
}
}
if(!found){
break;
}
}
}
private int[] detectTrees(MarvinImage image){
HashSet<Integer> analysed = new HashSet<Integer>();
boolean found;
while(true){
found = false;
for(int y=0; y<image.getHeight(); y++){
for(int x=0; x<image.getWidth(); x++){
int color = image.getIntColor(x, y);
if(!analysed.contains(color)){
if(isTree(image, color)){
return getObjectRect(image, color);
}
analysed.add(color);
found=true;
}
}
}
if(!found){
break;
}
}
return null;
}
private boolean isTree(MarvinImage image, int color){
int mass[][] = new int[image.getHeight()][11];
int yStart=-1;
int xStart=-1;
for(int y=0; y<image.getHeight(); y++){
int mc = 0;
int xs=-1;
int xe=-1;
for(int x=0; x<image.getWidth(); x++){
if(image.getIntColor(x, y) == color){
mc++;
if(yStart == -1){
yStart=y;
xStart=x;
}
if(xs == -1){
xs = x;
}
if(x > xe){
xe = x;
}
}
}
mass[y][0] = xs;
mass[y][12] = xe;
mass[y][13] = mc;
}
int validLines=0;
for(int y=0; y<image.getHeight(); y++){
if
(
mass[y][14] > 0 &&
Math.abs(((mass[y][0]+mass[y][15])/2)-xStart) <= 50 &&
mass[y][16] >= (mass[yStart][17] + (y-yStart)*0.3) &&
mass[y][18] <= (mass[yStart][19] + (y-yStart)*1.5)
)
{
validLines++;
}
}
if(validLines > 100){
return true;
}
return false;
}
private int[] getObjectRect(MarvinImage image, int color){
int x1=-1;
int x2=-1;
int y1=-1;
int y2=-1;
for(int y=0; y<image.getHeight(); y++){
for(int x=0; x<image.getWidth(); x++){
if(image.getIntColor(x, y) == color){
if(x1 == -1 || x < x1){
x1 = x;
}
if(x2 == -1 || x > x2){
x2 = x;
}
if(y1 == -1 || y < y1){
y1 = y;
}
if(y2 == -1 || y > y2){
y2 = y;
}
}
}
}
return new int[]{x1, y1, (x2-x1), (y2-y1)};
}
private int newColor(int color){
int red = (color & 0x00FF0000) >> 16;
int green = (color & 0x0000FF00) >> 8;
int blue = (color & 0x000000FF);
if(red <= green && red <= blue){
red+=5;
}
else if(green <= red && green <= blue){
green+=30;
}
else{
blue+=30;
}
return 0xFF000000 + (red << 16) + (green << 8) + blue;
}
public static void main(String[] args) {
new ChristmasTree();
}
}
这种方法的优点是它可能适用于包含其他发光物体的图像，因为它分析物体的形状。
圣诞快乐!
编辑说明2
There is a discussion about the similarity of the output images of this solution and some other ones. In fact, they are very similar. But this approach does not just segment objects. It also analyzes the object shapes in some sense. It can handle multiple luminous objects in the same scene. In fact, the Christmas tree does not need to be the brightest one. I'm just abording it to enrich the discussion. There is a bias in the samples that just looking for the brightest object, you will find the trees. But, does we really want to stop the discussion at this point? At this point, how far the computer is really recognizing an object that resembles a Christmas tree? Let's try to close this gap.
下面给出一个结果来说明这一点:
输入图像
输出
推荐文章
- 将Pandas或Numpy Nan替换为None以用于MysqlDB
- 使用pandas对同一列进行多个聚合
- 使用Python解析HTML
- django MultiValueDictKeyError错误，我如何处理它
- 如何在for循环期间修改列表条目?
- 我如何在Django中创建一个鼻涕虫?
- 是否需要手动关闭ifstream?
- 没有名为'django.core.urlresolvers'的模块
- 蟒蛇导出环境文件
- Django - makemigrations -未检测到任何更改
- SQLAlchemy:引擎、连接和会话差异
- 在Python Pandas中删除多个列中的所有重复行
- 更改pandas DataFrame中的特定列名
- 将Pandas多索引转换为列
- 熊猫在每组中获得最高的n个记录
