Public Documentation

Flow{T} = Matrix{Complex{T}} where T <: Real

Flow is a Matrix with elements that are Complex.

Image{T} = Matrix{T} where T

Image is a Matrix with elements that are Real.

addpoints(inds; ret::Symbol=:figure, labels=[])

Add points to a plot at the locations inds optionaly provide labels.

assess_flow_quality(flow, flow_est; title="", display::Bool=true)

Run a few tests on comparing flow_est and flow.

Tests: [angle_mae, angle_rmse, mae, mse]

assess_source_reg_quality(target, source_reg; title="", display::Bool=true)

Run a few tests on comparing target and source_reg.

Tests: [ncc, mae, rmse, mse]

find_edge_points(img::Image; spacing::Int=35, point_count::Int=35, sigma=1, mask::BitArray{2}=BitArray{2}(undef, 0,0))

Greadily locate point_count of points in img with the highest gradient magnitude. These points have will be atleast spacing pixels appart, they are returned as an array of CartesianIndex. A mask can be used to exclude some parts of img from the search.

Note: sigma is the gaussian filter used to smooth img before looking for gradients.

gen_anhir(base_path = "/Users/MrTrololord/Documents/anhir/";
          mutate=true,
          diag_pixels=500)

Generate a random ANHIR image pair target and source images (in this order) from the ANHIR dataset at base_path.

See also: load_anhir_image_pair

gen_chess(tile_size::Integer=50, board_size::Integer=4)

Create a chessboard image with board_size tiles in each dimension, where each tile is tile_size pixels in each dimension. Note that board_size has to be even.

See also: imgshow, gen_tiled_flow

gen_init(type::Symbol=:lena; flow_args=[])

Create the usual testing data; img, imgw, flow using the given parameters.

Arguments

• img_type::Symbol=:lena: what base image is used. [Options: :lena, :chess]
• flow_type::Symbol=:quad: what flow generation function is used. [Options: :tiled, :quad, :uniform, :spaghetti]

Keyword Arguments

• flow_args=[]: arguments passed to the flow generation function besides the flow size.
• img_args=[]: arguments passed to the img generation function if :chess is chosen.
• flipped::Bool=false: the image will be flipped along the horizontal axis, so that the origin is at the bottom.

Example

# chess image, warped chess image, flow with maximal displacement 20 generated by the `gen_quad_flow` function
img, imgw, flow = gen_init(:chess, :quad, flow_args=[20])

See also: [`gen_anhir`](@ref), [`gen_lena`](@ref), [`gen_spaghetti`](@ref), [`gen_tiled_flow`](@ref), [`gen_quad_flow`](@ref), [`gen_uniform_flow`](@ref)

gen_lena()

Get a 256x256 grayscale "lena" image.

gen_quad_flow(img_size, max_magnitude=20)

Generate a smoothly varying locally constant random flow of size img_size and with a maximal displacement of max_magnitude using a quadratic function:

where:

The constants a, b and c are random numbers from the normal distribution with mean 0 and standard deviation 1.

See also: showflow, gen_tiled_flow, Flow

gen_spaghetti(img_size = (200, 200),
              τ = 0.2,
              len = 30;
              count=25,
              spread=10)

Generate an image with count spaghetti images with the params τ (thickness), len (length) and spread. The coordinates x and y in the equation are scaled from (-1, 1) and the spread gives the length of interval in which the (-1, 1) is randomly located. The genereated spaghetti is also randomly rotated.

(Example for spread 10, the interval for x and y is randomly anywhere between (-9:1) and (-1:9))

Example

# feature-full spaghetti image
img = gen_spaghetti((256, 256), 0.15, 70, spread=20)
# feature-less spaghetti image
img = gen_spaghetti((256, 256), 1, 50, spread=12)

Note: For the exact intensity function see gen_one_spaghetti

See also: spaghetti_fun

gen_tiled_flow(flow_size::Tuple{T, T}=(200, 200),
max_magnitude::Real=20, tile_size=nothing; filter_amp=nothing)::Flow where {T <: Integer}

Generate a smoothly varying random flow. The flow parameters are set by the function arguments.

It works by generating a tiled flow where each tile has a random uniform flow. Then it smooths these tiles by filtering with a gaussian.

Arguments

• flow_size::Tuple{T, T}=(200, 200): dimensions of the flow.
• max_magnitude::Real=20: maximum allowed amplitude of the displacement.
• tile_size=nothing: size of random uniform flow tiles that make up the start matrix. (The larger the slower the flow.) Note: If set to flow_size or larger it will generate a uniform pixel shift in a random direction.
• filter_amp=nothing: size of the gaussian filter which is used to smooth the random start matrix.

See also: showflow, Flow, gen_chess, gen_quad_flow

gen_uniform_flow(flow_size=(200, 200), vector=1 + 1im, max_magnitude=vec_len(vector))

Generate a uniform flow of size flow_size, where every displacement vector is vector, scaled by max_magnitude.

Note: max_magnitude = vec_len(vector) by default, so if nothing is enetered as max_magnitude the flow is made up of vector and is not scaled.

See also: showflow, gen_tiled_flow, Flow, gen_quad_flow

imgoverlay(img1, img2; fig=nothing, figtitle::String="Image overlay", ret::Symbol=:figure)

Show an overlay of the images img1 and img2, each in a different colors, to better see the differences between them.

See also: imgshow

imgshow(img; <keyword arguments>)

Return a figure with image img. Plot origin is in the botom left.

Keyword Arguments

• fig=nothing: add a figure to plot in. By defaults creates a blank new figure.
• figtitle::String="Image": add title to the figure.
• ret::Symbol=:figure: set return object, by default returns Figure, other options: :pyobject returns a PyObject. (Using figure makes Juno directly plot.)
• origin_bot::Bool=false: set the origin to the lower left corner, which is not typical for images (they will appear flipped).

See also: showflow, imgshowflow, warp_imgshowflow

imgshowflow(img, flow; <keyword arguments>)

Return a figure with an image img and displacement field flow.

Keyword Arguments

• img: the image to be plotted.
• flow::Flow: the vector flow to be plotted.
• disp_type::Symbol=:full : display mode, either :full -> display all the vectors, or :sparse -> displays only the vectors above a threshold.
• skip_count=nothing: the number of vectors to skip between each displayed vector. By default set so that the output is $20 × 20$ vectors.
• fig=nothing: add a figure to plot in. By defaults creates a blank new figure.
• mag::Real=1: magnify the plotted vectors.
• key::Bool=true: add key with maximum vector length.
• figtitle::String="Iamge with Flow": add title to the figure.
• ret::Symbol=:figure: set return object, by default returns Figure, other options: :pyobject returns a PyObject. (Using figure makes Juno directly plot.)
• origin_bot::Bool=true: set the origin to the lower left corner, which is not typical for images (they will appear flipped).

See also: imgshow, showflow, warp_imgshowflow

interpolateflow(flowatinds, inds::Array{CartesianIndex, 1}, flowsize; method::Symbol=:quad, kwargs=Dict())

Interpolate a complex dispalcment field of size flow_size from an array of displacement vectors flow_at_inds, that are at the locations inds. You can choose between two methods method=:quad which fits the displacement vectors into a global quadratic displacement field and method=:rbf which uses an rbf model. Additional keyword arguments of the chosen method can be passed as kwargs.

See also: Flow, interpolate_flow_quad, interpolate_flow_rbf

interpolate_flow_quad(flow_at_inds, inds, flow_size)

Interpolate a complex displacement field of size flow_size from displacement vectors flow_at_inds at inds using a global quadratic model.

The fit to this model is made by minimizing the following:

,

where:

See also: showflow, Flow, interpolate_flow, interpolate_flow_rbf

lap(args; kwargs)

Return a 2D Flow matrix of displacements that transforms source closer to target and source image transformed with this displacement field, source_reg.

Details

Perform the classic Local All-Pass algorithm with post-proccessing (inpainting and smoothing), then warps the source image with the resulting estimate flow.

Arguments:

• target::Image: target/fixed grayscale image.
• source::Image: source/moving grayscale image.
• fhs::Integer: the half size of the base of the gaussian filters used.
• window_size=(2fhs+1, 2fhs+1): the size of the local window (tuple of 2 ints), usually same as filter_size.

Keyword Arguments:

• timer::TimerOutput=TimerOutput("lap"): provide a timer which times certain blocks in the function.
• display::Bool=false: verbose and debug prints.

Outputs:

• flow_estim: full estimated flow.
• source_reg: source image warped by flow_estim.

See also: single_lap, inpaint_nans!, smooth_with_gaussian!, warp_img.

load_anhir_image_pair(target_path,
                      source_path;
                      base_path = "/Users/MrTrololord/Documents/anhir/dataset")

Load an image pair in the locations target_path and source_path from the location base_path.

See also: gen_anhir

function mean(x)

Calculate the mean of x.

pflap(args; kwargs)

Find a transformation flow (complex displacment field), that transforms image source to image target. Returns the transformation a Flow and the registered source image.

Arguments

• target::Image: the image we want source to look like.
• source::Image: warped image we want to transform into target.

Keyword Arguments

• filter_count::Integer=3: the number of basis filters used in single_lap calls (so far only =3 implemented).
• max_repeats::Integer=1: the maximum number of times an iteration of one filter size can be repeated.
• display::Bool=true: use verbose prints and return an array of figures.
• timer::TimerOutput=TimerOutput("pflap"): provide a timer which times certain blocks in the function.
• match_source_histogram::Bool=true: choose to match the source image histogram to the target image histogram.
• rescale_intensities::Bool=false: choose to rescale! intensities of the images to the whole [0, 1] spectrum.

Outputs

• flow::Flow: is the complex vector field that transforms source closer to target.
• source_reg::Image: is the image source transformed by flow.
• [figs::Matrix{Figure}: is a 2D array of PyPlot Figures which shows the work of the algorithm at each iteration. For each iteration there are 3 Figures in this order: 1) current u_est, 2) newest addition to u_est Δ_u, 3) current source_reg.]

Describtion

Implements the basic concept of Algorithm 2 from the paper without some features. It uses single_lap iteratively; in each iteration using the transformation estimated by single_lap to warp the source image closer to the target image and then using this warped closer image as the source image in the next iteration, while using progressively smaller filter_half_sizes to estimate even small and faster varying displacements.

Note:

If display=true is chosen the function will also return an array of figures demonstrating the work of the registration.

See also: single_lap, imgshow,imgshowflow, warp_imgshowflow, Flow

resize_to_diag_size(img, desired_diag_size)

Resize img so that it has desired_diag_size pixels on its diagonal.

See also: gen_anhir, load_anhir_image_pair

showflow(flow::Flow; <keyword arguments>)

Return a figure with the displacement field flow by default skipping some vectors to make it easy to read.

Keyword Arguments

• flow::Flow: the vector flow to be plotted.
• disp_type::Symbol=:full : display mode, either :full -> display all the vectors, or :sparse -> displays only the vectors above a threshold, or :auto which decide for you based on the data.
• skip_count=nothing: the number of vectors to skip between each displayed vector. By default set so that the output is $20 × 20$ vectors.
• fig=nothing: add a figure to plot in. By defaults creates a blank new figure.
• mag::Real=1: magnify the plotted vectors.
• key::Bool=true: add key with maximum vector length.
• figtitle::String="Flow": add title to the figure.
• ret::Symbol=:figure: set return object, by default returns Figure, other options: :pyobject returns a PyObject. (Using figure makes Juno directly plot.)

See also: imgshow, imgshowflow, warp_imgshowflow

single_lap(target::Image, source::Image, filter_half_size::Integer, window, filter_count::Integer=3)

Return an estimate of a smoothly varying flow of size of target which is the displacement that transforms source closer to target.

Arguments:

• target::Image: target/fixed grayscale image.
• source::Image: source/moving grayscale image.
• filter_half_size: the half size of the base of the gaussian filters used.
• window: the size of the local window (list of 2 ints) usually same as filter_size.
• filter_count::Integer=3: the number of basis filters used (so far only =3 implemented).

Keyword Arguments:

• timer::TimerOutput=TimerOutput("LAP"): provide a timer which times certain blocks in the function.
• display::Bool=false: verbose and debug prints.

See also: pflap, showflow, warp_imgshowflow, imgshowflow

single_lap_at_points(args; kwargs)

Return a vector of displacements that transforms source closer to target at points inds.

Arguments:

• target::Image: target/fixed grayscale image.
• source::Image: source/moving grayscale image.
• filter_half_size::Integer: the half size of the base of the gaussian filters used.
• window: the size of the local window (list of 2 ints) usually same as filter_size.
• inds::Array{CartesianIndex{2},1}: where to calculate the flow.

Keyword Arguments:

• filter_count::Integer=3: the number of basis filters used (so far only =3 implemented).
• timer::TimerOutput=TimerOutput("Sparse LAP"): provide a timer which times certain blocks in the function.
• display::Bool=false: verbose and debug prints.

See also: pflap, showflow, single_lap, sparse_pflap.

sparse_lap(args; kwargs)

Return a 2D Flow matrix of displacements that transforms source closer to target.

Details

Uses the Local All-Pass algorithm at indices, inds, of the target image, with a high gradient magnitude, to estimate displacement vectors at these locations. These estimated vectors are then fit into a parametric model (flow_interpolation_method) and a full displacemnt field/flow, or full_flow_estim, is made. This flow is then used to warp the source image, creating the source_reg image.

Arguments:

• target::Image: target/fixed grayscale image.
• source::Image: source/moving grayscale image.
• fhs::Integer: the half size of the base of the gaussian filters used.
• window_size=(2fhs+1, 2fhs+1): the size of the local window (tuple of 2 ints), usually same as filter_size.

Keyword Arguments:

• spacing::Integer=25: the smallest number of pixels that can separate two inds. See also: find_edge_points
• point_count::Integer=100: the number of inds we are looking for.

(The method will use the maximum amount up to this number if the this amount of inds isn't found.) See also: find_edge_points

• timer::TimerOutput=TimerOutput("sparse_lap"): provide a timer which times certain blocks in the function.
• display::Bool=false: verbose and debug prints.
• flow_interpolation_method::Symbol=:quad: choose which strategy to use for sparse flow interpolation.

(Choices: :quad -> fits to a global quadratic model, :rbf -> fits to a local rbf model.) See also: interpolate_flow

• base_method_kwargs=Dict(:timer => timer, :display => display)): keyword arguments passed to the base method.

(In this case: single_lap_at_points).

Outputs:

• full_flow_estim: full estimated flow.
• source_reg: source image warped by full_flow_estim. See also: warp_img.
• flow_estim_at_inds: estimated displacement vectors, at the indices inds. These are fitted into the flow_interpolation_method to

make the full estimated flow full_flow_estim.

• inds: indices chosen for the high gradient at which the flow_estim_at_inds is estimated.

See also: showflow, imgshow, find_edge_points, interpolate_flow

sparse_pflap(args; kwargs)

Find a transformation flow (complex displacment field), that transforms image source to closer to image target. Returns the transformation a Flow and the registered source image.

Arguments:

• target::Image: target/fixed grayscale image.
• source::Image: source/moving grayscale image.

Keyword Arguments:

• filter_count::Integer=3: the number of basis filters used (so far only =3 implemented).
• timer::TimerOutput=TimerOutput("sparse pflap"): provide a timer which times certain blocks in the function.
• display::Bool=false: verbose and debug prints.
• point_count::Int=500: the number of points attempted to be found at the edges of the target image.
• spacing::Int=10: the minimal distance between two points.
• match_source_histogram::Bool=true: choose to match the source image histogram to the target image histogram.

Note:

If display=true is chosen the function will also return an array of figures demonstrating the work of the registration.

See also: sparse_lap, pflap, showflow, single_lap, sparse_pflap.

sparse_pflap(args; kwargs)

Find a transformation flow (complex displacment field), that transforms image source to closer to image target. Returns the transformation a Flow and the registered source image.

Arguments:

• target::Image: target/fixed grayscale image.
• source::Image: source/moving grayscale image.

Keyword Arguments:

• filter_count::Integer=3: the number of basis filters used (so far only =3 implemented).
• timer::TimerOutput=TimerOutput("sparse pflap"): provide a timer which times certain blocks in the function.
• display::Bool=false: verbose and debug prints.
• point_count::Int=500: the number of points attempted to be found at the edges of the target image.
• spacing::Int=10: the minimal distance between two points.
• match_source_histogram::Bool=true: choose to match the source image histogram to the target image histogram.

Note:

If display=true is chosen the function will also return an array of figures demonstrating the work of the registration.

See also: sparse_lap, pflap, showflow, single_lap, sparse_pflap.

test_registration_alg(args; kwargs)

Test a registration function method by timing it and comparing its outputs to the ground truth flow and the target image.

Arguments:

• method: registration function.
• target::Image: target image.
• source::Image: source image to be warped closer to target.
• flow::Flow: truth displacement flow warping target to source.

Keyword Arguments:

• display::Bool=true: print debugging info and test results.
• timer::TimerOutput=TimerOutput("blank"): timer to for certain blocks of code in the method.
• method_args=[]: arguments for the registration function.
• method_kwargs::Dict=Dict(): keyword arguments for the registration function.

Outputs:

• flow_est: estimated transformation flow.
• source_reg: source image registered.
• timer: timings of the method insides.
• results: Dict with results of the quality of the registration.
• [(outputs)]: other outputs of the method besides flow_est and source_reg.

Example

# generate a lena image, a warped lena image and a random quadratic flow
img, imgw, flow = gen_init();
# setup timer
timer=TimerOutput("sparse pf lap");
# choose method params
method_kwargs = Dict(:timer => timer, :display => false, :max_repeats => 1, :point_count => 500, :spacing => 10)
# run and test `sparse_pflap`
flow_est, source_reg, timer, results = test_registration_alg(sparse_pflap, img, imgw, flow, [], method_kwargs, timer=timer)

See also: lap, sparse_lap,pflap, sparse_pflap, Flow, time_reg_alg

time_reg_alg(args; kwargs)

Run a registration method and get the time of execution and method outputs.

Arguments:

• method: registration function.
• target::Image: target image.
• source::Image: source image to be warped closer to target.

Keyword Arguments:

• method_args=[]: arguments for the registration function.
• display::Bool=true: print debugging info and test results.
• timer::TimerOutput=TimerOutput("blank"): timer to for certain blocks of code in the method.
• method_kwargs::Dict=Dict(:timer => timer, :display => display): keyword arguments for the registration function.

Outputs:

• flow_est: estimated transformation flow.
• source_reg: source image registered.
• timer: timings of the method insides.
• time_in_secs: execution time of the method in seconds.
• [(outputs)]: other outputs of the method besides flow_est and source_reg.

See also: test_registration_alg

warp_img(img::Image, dx, dy; border_strat::Symbol=:replicate)::Image
warp_img(img::Image, dx, dy, filling_img::Image)::Image

Warp the image img by dx in x direction and by dy in y direction. border_strat indicates how to act when the resulting coordinate outside of bounds of img.

Possible variations:

• border_strat == :replicate: fill with the closest pixel value.
• border_strat == :zeros: fill with zeros.
• supply a filling_img as 3rd argument: fill with the provided image filling_img

See also: showflow, imgshowflow, Flow

warp_imgshowflow(img, flow; <keyword arguments>)

Return a figure with an image and a displacement field, where the image is warped by the displacement field.

Keyword Arguments

• img: the image to be plotted.
• flow::Flow: the vector flow to be plotted.
• disp_type::Symbol=:full : display mode, either :full -> display all the vectors, or :sparse -> displays only the vectors above a threshold.
• skip_count=nothing: the number of vectors to skip between each displayed vector. By default set so that the output is $20 × 20$ vectors.
• fig=nothing: add a figure to plot in. By defaults creates a blank new figure.
• mag::Real=1: magnify the plotted vectors.
• key::Bool=true: add key with maximum vector length.
• figtitle::String="Iamge with Flow": add title to the figure.
• ret::Symbol=:figure: set return object, by default returns Figure, other options: :pyobject returns a PyObject. (Using figure makes Juno directly plot.)
• origin_bot::Bool=true: set the origin to the lower left corner, which is not typical for images (they will appear flipped).

See also: imgshow, showflow, imgshowflow