Implementing image processing algorithms in VHDL is a scary thing for many. Though I agree that its much more difficult to do it in VHDL than in a high level programming language like C, Matlab etc, it needn't be that scary.
In this post I am going to share the code for a simple image processing algorithm - A RGB to Gray scale image converter.
There are many ways, from simple to complex, in which you can do this. I have done it in a way, which makes sense to me. Touching upon few topics related to this subject. For example reading the image data from a text file, storing it in RAM and accessing the data within the code and then manipulating them etc...
I have used the standard Matlab image Lenna.bmp for this. The original image was 512*512*3 pixels in size. This takes a long time to load and run in Modelsim. So I first reduced its size to 1/8th of its original size, making it a 64*64*3 pixel image. Each pixel ranges from 0 to 255 and the dimension "3" indicates the presence of Red, Green and Blue components.
VHDL text file operations aren't ideal for reading multiple pixels from the same row. So I converted the 3 Dimensional image data into a 1 Dimensional matrix. And then used dlmwrite Matlab command to write it to a text file named rgb.txt. This will be our input image.
In Matlab, I manually converted the above RGB image into a gray scale image using the formula:
Grayscale Image = ( (0.3 * R) + (0.59 * G) + (0.11 * B) ).
The above grayscale pixels were converted to a 1-D matrix and written to a text file called gray.txt. This text file would be read by our VHDL testbench to verify that the results from our VHDL design is the same as the ideal result obtained from Matlab.
The Matlab program which I used for achieving all this is shared below:
I=imread('Lenna.bmp'); %read the
image into memory
I=imresize(I,1/8); %reduce the
size by 8 times.
%convert image to 1-D array and write it to a text file.
dlmwrite('rgb.txt',reshape(I,64*64*3,1,1));
I4=double(I); %convert it to
double format
%convert rgb pixels to gray manually as per formula.
for i=1:64
for j=1:64
I2(i,j) =
I4(i,j,1)*0.3 + I4(i,j,2)*0.59 + I4(i,j,3)*0.11;
end
end
%the converted image is changed to 1-D and then written to a
text file.
I3=reshape(I2,64*64,1);
dlmwrite('gray.txt',I3);
The standard RGB format Lenna image, the resized version of it and the converted Grayscale image are shared below:
Now that we have prepared the above basic data to work with, we can go and explore the VHDL designs. There are three VHDL files in this project:
1) im_ram.vhd:
The input RGB image is internally stored in a RAM. The RAM is declared and initialized with the pixel values from the text file rgb.txt. The std.textio library is used, to do the file reading operations.
--RAM entity for storing the image.
library ieee;
use ieee.std_logic_1164.all;
use ieee.numeric_std.all;
--the below libraries are needed for reading text file.
library std;
use std.textio.all;
entity im_ram is
generic(
ADDR_WIDTH : integer := 16; --Address bus size of the Image Ram.
IM_SIZE_D1 : integer := 64; --Size along Dimension 1
IM_SIZE_D2 : integer := 64 --Size along Dimension 2
);
port (
Clk : in std_logic;
addr_in : in unsigned(ADDR_WIDTH-1 downto 0); --Address bus to the Image Ram.
rgb_out : out std_logic_vector(23 downto 0) --24 bit RGB pixel output
);
end im_ram;
architecture Behav of im_ram is
--custom array declaration.
type im_ram_type is array (0 to IM_SIZE_D1*IM_SIZE_D2-1) of std_logic_vector(23 downto 0);
--function for reading the image pixels from text file and use
--it to initialize the RAM.
impure function im_ram_initialize return im_ram_type is
variable line_var : line;
file text_var : text;
variable pixel : integer;
variable image_pixels : im_ram_type;
begin
--Open the file in read mode.
file_open(text_var,"rgb.txt",read_mode);
while(NOT ENDFILE(text_var)) loop --until end of file is reached
for k in 1 to 3 loop --through R, G and B.
for j in 0 to IM_SIZE_D2-1 loop
for i in 0 to IM_SIZE_D1-1 loop
readline(text_var,line_var); --read one row. Each row contains one pixel.
read(line_var,pixel); --From the read line, read the integer value.
--save the pixel in the RAM.
image_pixels(i*IM_SIZE_D2+j)(k*8-1 downto k*8-8) := std_logic_vector(to_unsigned(pixel,8));
end loop;
end loop;
end loop;
end loop;
file_close(text_var); --close the file after reading.
return image_pixels;
end function;
--declare and initialize the image ram.
signal ram : im_ram_type := im_ram_initialize;
begin
--read the R,G and B pixels from RAM with the addr_in input.
rgb_out <= ram(to_integer(addr_in));
end architecture;
2) rgb2gray.vhd:
This is the top level entity which converts the image from RGB format to Grayscale. The RAM block is instantiated as a component inside this entity.
--Convert a internally stored RGB image into gray image.
library ieee;
use ieee.std_logic_1164.all;
use ieee.numeric_std.all;
entity rgb2gray is
generic(
ADDR_WIDTH : integer := 16; --Address bus size of the Image Ram.
IM_SIZE_D1 : integer := 64; --Size along Dimension 1
IM_SIZE_D2 : integer := 64 --Size along Dimension 2
);
port (
Clk : in std_logic;
reset : in std_logic; --active high asynchronous reset
data_valid : out std_logic; --High when gray_out has valid output.
gray_out : out unsigned(7 downto 0) --8 bit gray pixel output
);
end rgb2gray;
architecture Behav of rgb2gray is
component im_ram is
generic(
ADDR_WIDTH : integer := 16; --Address bus size of the Image Ram.
IM_SIZE_D1 : integer := 64; --Size along Dimension 1
IM_SIZE_D2 : integer := 64 --Size along Dimension 2
);
port (
Clk : in std_logic;
addr_in : in unsigned(ADDR_WIDTH-1 downto 0); --Address bus to the Image Ram.
rgb_out : out std_logic_vector(23 downto 0) --24 bit RGB pixel output
);
end component;
signal rgb_out : std_logic_vector(23 downto 0);
signal addr_in : unsigned(ADDR_WIDTH-1 downto 0);
begin
--Instantiation of Image RAM. Internally stored image.
image_ram : im_ram generic map(ADDR_WIDTH, IM_SIZE_D1 ,IM_SIZE_D2)
port map(Clk, addr_in, rgb_out);
--Process to convert RGB to Gray image.
CONVERTER_PROC : process(Clk,reset)
--temperary variables
variable temp1,temp2,temp3,temp4 : unsigned(15 downto 0);
begin
if(reset = '1') then --active high asynchronous reset
addr_in <= (others => '0');
data_valid <= '0';
elsif rising_edge(Clk) then
--output is ready when the last address in the ram has reached.
if(to_integer(addr_in) = IM_SIZE_D1*IM_SIZE_D2-1) then
addr_in <= (others => '0');
data_valid <= '0';
else --otherwise keep incrementing the address value.
addr_in <= addr_in + 1;
data_valid <= '1'; --indicates output is ready
end if;
--Gray pixel = 0.3*Red pixel + 0.59*Green pixel + 0.11*Blue pixel
--the 24 bit value is split into R,G and B components and multiplied
--with their respective weights and then added together.
temp1 := "01001100" * unsigned(rgb_out(7 downto 0)); --(0.3 * R)
temp2 := "10010111" * unsigned(rgb_out(15 downto 8)); --(0.59 * G)
temp3 := "00011100" * unsigned(rgb_out(23 downto 16)); --(0.11 * B)
temp4 := temp1 + temp2 + temp3;
--Most significant bit of the LSB portion is added to the MSB portion.
--To round off the result.
gray_out <= temp4(15 downto 8) + ("0000000" & temp4(7));
end if;
end process;
end architecture;
3) tb.vhd:
This is the testbench for testing our main designs. The output image is received from the main design and compared with the actual result. Previously using Matlab, we have saved the actual result in a file called gray.txt .
--Testbench
library ieee;
use ieee.std_logic_1164.all;
use ieee.numeric_std.all;
--the below libraries are needed for reading text file.
library std;
use std.textio.all;
--Testbench entity is always empty.
entity tb is
end tb;
architecture sim of tb is
--the generic parameters are declared and initialized as constants here.
constant IM_SIZE_D1: integer := 64;
constant IM_SIZE_D2: integer := 64; --we have a 64*64 image.
constant ADDR_WIDTH: integer := 12; --64*64=4096 which needs 12 bits.
--this array is used to store the output gray pixels.
type image_type is array (1 to IM_SIZE_D1, 1 to IM_SIZE_D2) of integer;
signal image_pixels : image_type := (others => (others => 0));
--component declaration.
component rgb2gray is
generic(
ADDR_WIDTH : integer := 16;
IM_SIZE_D1 : integer := 64;
IM_SIZE_D2 : integer := 64
);
port (
Clk : in std_logic;
reset : in std_logic;
data_valid : out std_logic;
gray_out : out unsigned(7 downto 0)
);
end component;
--temperory signal declarations.
signal Clk,reset,data_valid : std_logic := '0';
signal gray_out : unsigned(7 downto 0);
constant Clk_period : time := 10 ns; --clock period.
begin
--generate the clock signal.
Clk <= not Clk after Clk_period/2;
--Instantiate the Unit under test.
UUT : rgb2gray generic map(ADDR_WIDTH, IM_SIZE_D1, IM_SIZE_D2)
port map(Clk, reset, data_valid, gray_out);
--Process where we apply inputs, read outputs and verify the result.
STIMULUS_PROC : process
variable i,j : integer := 1; --loop indices.
variable line_var : line;
file text_var : text;
variable pixel : integer;
variable error : integer := 0; --this value should be zero at the end of simulation.
variable diff : image_type := (others => (others => 0));
begin
reset <= '1';
wait for Clk_period;
reset <= '0'; --reset is applied for one clock cycle.
wait until data_valid = '1'; --wait for valid data at the output port.
while(data_valid = '1') loop
wait until (falling_edge(Clk)); --sample outputs at the falling edge of clock
image_pixels(i,j) <= to_integer(gray_out); --save pixel as integer.
--generate indices to save the pixels in the correct place.
if(j = IM_SIZE_D2) then
j := 1;
if(i = IM_SIZE_D1) then
i := 1;
else
i := i+1;
end if;
else
j := j+1;
end if;
wait until (rising_edge(Clk)); --pause until rising edge of the clock
end loop;
--all output gray pixels are read. Activate reset again.
reset <= '1';
--Now check if the results are the same as in Matlab
--Open the file in read mode. gray.txt contains pixels calculated using Matlab.
file_open(text_var,"gray.txt",read_mode);
while(NOT ENDFILE(text_var)) loop --until end of file is reached
for j in 1 to IM_SIZE_D2 loop
for i in 1 to IM_SIZE_D1 loop
readline(text_var,line_var); --read one row. Each row contains one pixel.
read(line_var,pixel);
--calculate the difference between actual gray pixels from Matlab
--and pixel values from our rgb2gray VHDL design.
diff(i,j) := abs(image_pixels(i,j) - pixel);
--If the difference is 2 or more then, we take it as an error
--and increment the variable 'error' by 1.
if(diff(i,j) > 1) then
error := error+1;
end if;
wait for 1 ns; --pause for 1 ns.
end loop;
end loop;
end loop;
file_close(text_var); --close the file after reading.
wait; --wait eternally after finishing simulation.
end process;
end architecture; --End of Testbench
Note that IM_SIZE_D1 and IM_SIZE_D2 are the number of rows and columns respectively of the image matrix. You can read it as a short form for "Image size along dimension 1 (or) 2".
In the testbench we have a variable matrix called "diff". This matrix contains the absolute value of difference between gray image pixels, from Matlab and VHDL. A difference of "1" is considered okay, as binary multiplications can cause some loss of least significant bits. But more than a difference of "1" is considered as an error. We can see that at the end of simulation, we didn't get any "error".
There is no address output from the top level entity rgb2gray. In our design, I thought it was unnecessary, because the pixel outputs were coming out in a sequential manner. So the testbench "knew" where to place each output pixel.
I hope this post was helpful for you and gave you some ideas on how to do image processing using VHDL.
You can Download the VHDL codes, images etc from Here.
