Rebol/OpenCV

Initial implementation of the OpenCV extension for Rebol3 (3.14.0 or newer).

So far it is considered just like a proof of concept and tested only on macOS with OpenCV installed using homebrew!

When building on macOS, the setup expects, that OpenCV includes and libs are accessible. So it is recommanded to use something like:

ln -s /opt/homebrew/Cellar/opencv/4.6.0/include/opencv4/opencv2 /usr/local/include/opencv2
ln -s /opt/homebrew/Cellar/opencv/4.6.0/lib /usr/local/lib/opencv

To import the extension from Rebol on macOS, the Rebol must be signed with entitlemens containing com.apple.security.cs.disable-library-validation as true.

Feature requests are welcome.

For list of currently supported extension commands and other values, read Commands.md.

Usage

All folowing examples expect, that OpenCV extension was imported using one of these methods:

1. Using direct path to the file: cv: import %path/to/opencv.rebx
2. Using extension in the default location: cv: import 'opencv

Making a blank matrix and displaying it.

Matrices are one of the main datatypes used by OpenCV extension.

mat: cv/Matrix 250x140 ;; creates an empty black image handle
cv/imshow mat          ;; open a window with default name "Image" displaying the image
cv/waitKey 0           ;; and wait for any key

Closing a window

Window is closed using destroyWindow "Window name". It is possible to close all windows using destroyAllWindows

cv/destroyWindow "Image"       ;; because "Image" is the default window's name.

Loading a matrix from file

Instead of writting full paths to cv commands, like: cv/imshow, it is possible to bind the code to the cv context using with cv [...]

with cv [
    filename: %image/mask.png
    mat: imread filename
    imshow/name mat filename   ;; displaying the image in the window with file name's title
    waitKey 0
]

Moving/resizing an opened window

Having the window from the previous example still open, it is possible to move it using moveWindow.

cv/moveWindow filename 300x50
cv/waitKey 5000                ;; now there is only 5s wait time

Windows created using namedWindow may be resized using resizeWindow

cv/namedWindow win: "Resized"  ;; creating a window with title/name "Resized"
cv/imshow/name mat win         ;; displaying an image in it
cv/resizeWindow win 500x280    ;; resized
cv/waitKey 5000
cv/destroyAllWindows           ;; closing both windows

Getting matrix properties

Using still the matrix from above, resized using resize command so the output is not too long..

with cv [
    mat: resize mat 10%
    probe get-property mat MAT_SIZE     ;; image size
    probe get-property mat MAT_TYPE     ;; CV type id
    probe get-property mat MAT_CHANNELS ;; number of channels (3 = RGB)
    probe get-property mat MAT_DEPTH    ;; CV depth id (0 = CV_8U)
    probe get-property mat MAT_BINARY   ;; raw binary data
    probe get-property mat MAT_VECTOR   ;; Rebol vector value
    probe get-property mat MAT_IMAGE    ;; Rebol image value
]

Above is now also possible with direct getters, like:

with cv [
    mat: resize mat 10%
    print ["Image size:"         mat/size     ]
    print ["CV type:"            mat/type     ]
    print ["Channels:"           mat/channels ]
    print ["CV depth:"           mat/depth    ]
    print ["Rebol binary data:"  mat/binary   ]
    print ["Rebol vector value:" mat/vector   ]
    print ["Rebol image value:"  mat/image    ]
]

Manually releasing matrices

Normally matrices are automatically released by Rebol's GC, but it is also possible to free them manually

cv/free mat          ;; manually released matrix

It should be noted, that such a matrix is not usable anymore! This will fail:

probe cv/imshow mat  ;; will return false!

Color space conversion

with cv [
    img: imread %image/mask.png
    hls:  cvtColor img none COLOR_BGR2HLS
    gray: cvtColor img none COLOR_BGR2GRAY
    namedWindow win1: "Original"
    namedWindow win2: "HLS"
    namedWindow win3: "Grayscale"
    imshow/name img  win1
    imshow/name hls  win2
    imshow/name gray win3
    moveWindow win1 0x0
    moveWindow win2 250x0
    moveWindow win3 500x0
    waitKey 0
]

Image threshold

Having the grayscale version from above, we can applie a fixed-level threshold.

with cv [
    namedWindow win4: "THRESH_BINARY"
    namedWindow win5: "THRESH_TRUNC"
    namedWindow win6: "THRESH_TOZERO"
    moveWindow  win4   0x150
    moveWindow  win5 250x150
    moveWindow  win6 500x150
    thresh1: threshold gray none  0  255 THRESH_BINARY
    thresh2: threshold gray none 100 255 THRESH_TRUNC
    thresh3: threshold gray none 200 255 THRESH_TOZERO
    imshow/name thresh1 win4
    imshow/name thresh2 win5
    imshow/name thresh3 win6
    waitKey 0
    destroyAllWindows
]

Using computed binary threshold as an opacity channel

image: cv/get-property img     cv/MAT_IMAGE   ;; get Rebol image
alpha: cv/get-property thresh1 cv/MAT_BINARY  ;; get Rebol binary with alpha values
image/alpha: alpha                            ;; replace image alpha with the new value
save %tmp/masked.png image                    ;; using Rebol's PNG codec to save the new image

Detecting edges in the image and their dilatation

with cv [
    src: imread "image/mask.png"
    namedWindow win1: "Canny"
    namedWindow win2: "Canny dilated"
    moveWindow win1 0x0
    moveWindow win2 250x0

    kernel: getStructuringElement MORPH_CROSS 3 -1 ;; preparing the kernel for dilatation 

    dst1: Canny src none 50 200                    ;; edge detecting
    dst2: dilate dst1 none kernel -1x-1 1          ;; dilating the edges
    imshow/name dst1 win1
    imshow/name dst2 win2
    waitKey 0
    destroyAllWindows
]

Detecting edges using Gabor Filter

with cv [
    lena: imread "image/lena.jpeg"
    size: get-property :lena MAT_SIZE
    gray: cvtColor :lena none COLOR_BGR2GRAY

    num-filters: 16
    sigma: 1.5    ;; The bandwidth or sigma controls the overall size of the Gabor envelope.
    theta: 0      ;; The theta controls the orientation of the Gabor function.
    lambd: 60     ;; The wavelength governs the width of the strips of the Gabor function.
    gamma: 0.75   ;; The aspect ratio or gamma controls the height of the Gabor function.
    psi: PI * 0.5 ;; The phase offset.

    theta-step: pi / num-filters
    kernels: make block! num-filters

    loop num-filters [
        ;- Generate new Gabor kernel
        kern: getGaborKernel 35x35 :sigma :theta :lambd :gamma :psi CV_64F
        normalize :kern :kern 1 0 NORM_L1 CV_64F ;- Brightness normalization
        ;- Store it for later use
        append kernels kern
        ;- Update the orientation for the next kernel
        theta: theta + theta-step
    ]

    temp: Matrix [:size CV_8UC1] ;; holds temporary filtered image
    dest: Matrix [:size CV_8UC1] ;; detected edges

    ;- Apply each Gabor kernel to filter the grayscale source
    foreach kern kernels [
        temp: filter2D :gray :temp -1 :kern -1x-1 0
        normalize :temp :temp 0 255 NORM_MINMAX
        ;- Compare our filter and cumulative image, taking the higher value
        max :dest :temp :dest
        ;- Display current detection state
        imshow/name :dest "Edges"
        waitKey 10
    ]
    waitKey 0
]

Blurring images

with cv [
    src: imread "image/mask.png"
    blured1: Matrix src
    blured2: Matrix src
    blured3: Matrix src
    namedWindow win1: "blur"
    namedWindow win2: "gaussianBlur"
    namedWindow win3: "medianBlur"
    moveWindow win1 0x0
    moveWindow win2 250x0
    moveWindow win3 500x0

    size: 1
    add-blur?: true
    forever [
        either add-blur? [
            size: size + 2
            add-blur?: size < 100
        ][
            size: size - 2
            add-blur?: size <= 1
        ]
        blur         :src :blured1 :size
        gaussianBlur :src :blured2 :size 0 0
        medianBlur   :src :blured3 :size
        imshow/name blured1 win1
        imshow/name blured2 win2
        imshow/name blured3 win3
        if 0 <= waitkey 50 [break]
    ]
    destroyAllWindows
]

Applying a color map

with cv [
    src: imread "image/lena.jpeg"
    result: Matrix :src
    for i 0 21 1 [
        applyColorMap :src :result :i 
        imshow :result
        setWindowTitle "Image" join "Colormap: " i
        if 0 < waitKey 1000 [break]
    ]
    destroyAllWindows
]

Applying Sepia filter to an image

with cv [
    src: imread "image/taj.jpg"
    ; using a binary for the kernel, but it should be possible
    ; to use vector directly later once implemented!
    kernel: #(float! [
        0.272 0.534 0.131
        0.349 0.686 0.168
        0.393 0.769 0.189
    ])
    sepia-filter: Matrix [3x3 :kernel]
    transform src src sepia-filter
    imshow src
    waitKey 0
    destroyAllWindows
]

Using shared buffer

When constructing matrices from a Rebol's binary or vector value, the buffer may be shared.

with cv [
    ;; allocate vector for a grayscale image of size 320x200
    data: #(uint8! 64000)
    ;; make an OpenCV metrix using the shared data
    img: Matrix [320x200 :data]
    ;; do some animation...
    forever [
        ;; manipulate the matrix data using Rebol code with the direct access
        forall data [data/1: random 255]
        ;; display the modified image using OpenCV
        imshow img
        if 0 < waitKey 10 [break]
    ]
]

Saving video from the camera

with cv [
    ;; initialize video input from a file...
    ;cam: VideoCapture %test.mp4
    ;; or from a camera device...
    cam: VideoCapture 0
    unless cam [print "Failed to initialize VideoCapture" quit]

    ;; resolve input frame size...
    size: as-pair get-property cam CAP_PROP_FRAME_WIDTH
                  get-property cam CAP_PROP_FRAME_HEIGHT

    print ["Input frame size:" size]

    ;set-property cam CAP_PROP_POS_FRAMES 2000.0 ;; can be used to set position in the video (file input)

    if frame: read :cam [
        ;; initialize VideoWriter (when 0 is used as codec parameter, than the output will be MJPG)
        out: VideoWriter %tmp/out.avi 0 24 size
        unless out [print "Failed to initialize VideoWriter!" quit]

        ;; grab 50 frames maximum...
        loop 50 [ 
            read/into :cam :frame    ;; reusing existing frame
            write out :frame         ;; append the frame to the output video
            imshow :frame            ;; and also show it in the window
            if pollKey = 27 [break]  ;; exit on ESC key
            wait 0.01                ;; let Rebol breath as well
        ]
        destroyAllWindows
    ]
    free :out  ;; release VideoWriter
    free :cam  ;; release VideoCapture
]

VideoWriter expects integer for its codec input. It is possible to use function like this for the conversion:

fourcc: func[
    "Converts fourcc codec integer to string and back"
    codec [any-string! binary! number!]][
    either number? codec [
        to string! reverse trim to binary! to integer! codec
    ][
        to integer! reverse to binary! codec
    ]
]
fourcc "avc1"    ;== 828601953
fourcc 828601953 ;== "avc1"

Computing absolute difference between 2 video frames

absdiff is useful when tracking a moving objects (or to produce nice psychedelic video effects:).

with cv [
    cam: VideoCapture 0
    unless cam [print "Failed to initialize VideoCapture" quit]
    if all [
        frame1: read :cam       ;; try to get the first frame
        frame2: Matrix :frame1  ;; make matrices with the same size and type
        result: Matrix :frame1  ;; for reuse later
    ][
        print "Press any key to quit."
        forever [ 
            read/into :cam :frame1          ;; get first frame
            if 0 < waitKey 50 [break]       ;; wait some time
            read/into :cam :frame2          ;; get second frame
            absdiff :frame1 :frame2 :result ;; compute absolute difference
            imshow :result                  ;; display it
            if 0 < waitKey 50 [break]       ;; for some time
        ]
        destroyAllWindows
    ]
    free :cam  ;; release VideoCapture
]

Encode and decode QR codes

with cv [
    mat: qrcode-encode "Hello Rebol, hello OpenCV!"
    mat: resize/with :mat 600% INTER_NEAREST
    ;; display the result...
    imshow :mat
    waitKey 0
    destroyAllWindows
    ;; save as PNG image...
    imwrite %tmp/test-qrcode.png :mat
    ;; decode QRCode from file:
    probe qrcode-decode %tmp/test-qrcode.png
]

---

This file was generated using examples.r3 script.