Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(lottie): add lottie widget #6081

Merged
merged 4 commits into from
May 23, 2024
Merged

Conversation

kisvegabor
Copy link
Member

@kisvegabor kisvegabor commented Apr 16, 2024

Description of the feature or fix

closes #6021

Overview

  • ThorVG is updated as there were Lottie related fixes in the newer versions
  • I've added the Lottie support as widget which is odd a little bit as it relies on a 3rd party lib

TODO:

  • support loading Lottie from file (ThorVG needs to updated to use LVGL's file system API)
  • consider replacing mallocs in ThorVG with lv_malloc
  • consider using lv_image_dsc_t or path in lv_lottile_set_src instead of havong 2 functions for data and path
  • try compiling ThorVG with Emscripten
  • fix the regression in image blending with ThorVG (cc @FASTSHIFT )

Notes

@kisvegabor
Copy link
Member Author

@FASTSHIFT With the updated ThorVG the cogwheel has some artifacts here. Do you have any idea why could it happen?

@IAMMX
Copy link
Contributor

IAMMX commented Apr 17, 2024

I think it's better to put Lottie in the others folder. Because widgets generally do not depend on 3rd party libraries.

@kisvegabor
Copy link
Member Author

I think it's better to put Lottie in the others folder. Because widgets generally do not depend on 3rd party libraries.

I was thinking the same way, however it's really like a widget from every aspect: has a class, widget-like API, treated as an lv_obj_t, etc.

I'm also afraid that in the others folders it will be less visible for the users.

This was referenced Apr 17, 2024
Copy link
Collaborator

@XuNeo XuNeo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Gabor,
How do you managed to upgrade the ThorVG lib? Is it done manually or there's script for help?

Is #if LV_USE_THORVG_INTERNAL needed for newly introduced source files?

This PR alone will fail on finding rlottie_capi.h.

@kisvegabor
Copy link
Member Author

How do you managed to upgrade the ThorVG lib? Is it done manually or there's script for help?

I've did it manually. This script helps to add the #if LV_USE_THORVG_INTERNAL guards.

This PR alone will fail on finding rlottie_capi.h.

How rottie is related? rlottie is a different lib, which should be deprecated once this PR (ThorVG Lottie) is merged.

@XuNeo
Copy link
Collaborator

XuNeo commented Apr 19, 2024

I have enabled the wrong config LV_USE_RLOTTIE. Now the code compiles but will crash.
@FASTSHIFT Could you help to check on your side?

ninja: Entering directory `build'
[660/661] cd /home/neo/projects/lvgl/lv_port_pc_eclipse/build && /home/neo/projects/lvgl/lv_port_pc_eclipse/bin/main
AddressSanitizer:DEADLYSIGNAL
=================================================================
==547919==ERROR: AddressSanitizer: SEGV on unknown address 0x57e26ea1d003 (pc 0x57e26e91e2a8 bp 0x7fffb05433b0 sp 0x7fffb0543360 T0)
==547919==The signal is caused by a READ memory access.
    #0 0x57e26e91e2a8 in argb8888_image_blend /home/neo/projects/lvgl/lv_port_pc_eclipse/lvgl/src/draw/sw/blend/lv_draw_sw_blend_to_rgb888.c:498
    #1 0x57e26e91d660 in lv_draw_sw_blend_image_to_rgb888 /home/neo/projects/lvgl/lv_port_pc_eclipse/lvgl/src/draw/sw/blend/lv_draw_sw_blend_to_rgb888.c:271
    #2 0x57e26e918f73 in lv_draw_sw_blend /home/neo/projects/lvgl/lv_port_pc_eclipse/lvgl/src/draw/sw/blend/lv_draw_sw_blend.c:143
    #3 0x57e26e9064ba in img_draw_core /home/neo/projects/lvgl/lv_port_pc_eclipse/lvgl/src/draw/sw/lv_draw_sw_img.c:235
    #4 0x57e26e8a4e0c in img_decode_and_draw /home/neo/projects/lvgl/lv_port_pc_eclipse/lvgl/src/draw/lv_draw_image.c:266
    #5 0x57e26e8a4850 in _lv_draw_image_normal_helper /home/neo/projects/lvgl/lv_port_pc_eclipse/lvgl/src/draw/lv_draw_image.c:165
    #6 0x57e26e9060ca in lv_draw_sw_image /home/neo/projects/lvgl/lv_port_pc_eclipse/lvgl/src/draw/sw/lv_draw_sw_img.c:164
    #7 0x57e26e900537 in execute_drawing /home/neo/projects/lvgl/lv_port_pc_eclipse/lvgl/src/draw/sw/lv_draw_sw.c:380
    #8 0x57e26e900254 in execute_drawing_unit /home/neo/projects/lvgl/lv_port_pc_eclipse/lvgl/src/draw/sw/lv_draw_sw.c:245
    #9 0x57e26e900432 in dispatch /home/neo/projects/lvgl/lv_port_pc_eclipse/lvgl/src/draw/sw/lv_draw_sw.c:325
    #10 0x57e26e8a200e in lv_draw_dispatch_layer /home/neo/projects/lvgl/lv_port_pc_eclipse/lvgl/src/draw/lv_draw.c:255
    #11 0x57e26e8a1cf7 in lv_draw_dispatch /home/neo/projects/lvgl/lv_port_pc_eclipse/lvgl/src/draw/lv_draw.c:161
    #12 0x57e26e8a1c5a in lv_draw_finalize_task_creation /home/neo/projects/lvgl/lv_port_pc_eclipse/lvgl/src/draw/lv_draw.c:138
    #13 0x57e26e8a457f in lv_draw_image /home/neo/projects/lvgl/lv_port_pc_eclipse/lvgl/src/draw/lv_draw_image.c:109
    #14 0x57e26e8e3bb7 in draw_image /home/neo/projects/lvgl/lv_port_pc_eclipse/lvgl/src/widgets/image/lv_image.c:764
    #15 0x57e26e8e354a in lv_image_event /home/neo/projects/lvgl/lv_port_pc_eclipse/lvgl/src/widgets/image/lv_image.c:664
    #16 0x57e26e88dcde in lv_obj_event_base /home/neo/projects/lvgl/lv_port_pc_eclipse/lvgl/src/core/lv_obj_event.c:86
    #17 0x57e26e88e89e in event_send_core /home/neo/projects/lvgl/lv_port_pc_eclipse/lvgl/src/core/lv_obj_event.c:359
    #18 0x57e26e88dc18 in lv_obj_send_event /home/neo/projects/lvgl/lv_port_pc_eclipse/lvgl/src/core/lv_obj_event.c:64
    #19 0x57e26e89bf6b in lv_obj_redraw /home/neo/projects/lvgl/lv_port_pc_eclipse/lvgl/src/core/lv_refr.c:110
    #20 0x57e26e89e042 in refr_obj /home/neo/projects/lvgl/lv_port_pc_eclipse/lvgl/src/core/lv_refr.c:892
    #21 0x57e26e89c163 in lv_obj_redraw /home/neo/projects/lvgl/lv_port_pc_eclipse/lvgl/src/core/lv_refr.c:161
    #22 0x57e26e89e042 in refr_obj /home/neo/projects/lvgl/lv_port_pc_eclipse/lvgl/src/core/lv_refr.c:892
    #23 0x57e26e89dc62 in refr_obj_and_children /home/neo/projects/lvgl/lv_port_pc_eclipse/lvgl/src/core/lv_refr.c:790
    #24 0x57e26e89da56 in refr_area_part /home/neo/projects/lvgl/lv_port_pc_eclipse/lvgl/src/core/lv_refr.c:723
    #25 0x57e26e89d5e3 in refr_area /home/neo/projects/lvgl/lv_port_pc_eclipse/lvgl/src/core/lv_refr.c:619
    #26 0x57e26e89d389 in refr_invalid_areas /home/neo/projects/lvgl/lv_port_pc_eclipse/lvgl/src/core/lv_refr.c:566
    #27 0x57e26e89cba8 in _lv_display_refr_timer /home/neo/projects/lvgl/lv_port_pc_eclipse/lvgl/src/core/lv_refr.c:374
    #28 0x57e26e8c46c4 in lv_timer_exec /home/neo/projects/lvgl/lv_port_pc_eclipse/lvgl/src/misc/lv_timer.c:300
    #29 0x57e26e8c3e48 in lv_timer_handler /home/neo/projects/lvgl/lv_port_pc_eclipse/lvgl/src/misc/lv_timer.c:105
    #30 0x57e26e882691 in main /home/neo/projects/lvgl/lv_port_pc_eclipse/main.c:1041
    #31 0x7519b5c29d8f in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
    #32 0x7519b5c29e3f in __libc_start_main_impl ../csu/libc-start.c:392
    #33 0x57e26e8802e4 in _start (/home/neo/projects/lvgl/lv_port_pc_eclipse/bin/main+0x122e4) (BuildId: 0cafb16ea873a52e22f08cd80a0b972eff635038)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /home/neo/projects/lvgl/lv_port_pc_eclipse/lvgl/src/draw/sw/blend/lv_draw_sw_blend_to_rgb888.c:498 in argb8888_image_blend
==547919==ABORTING
FAILED: CMakeFiles/run /home/neo/projects/lvgl/lv_port_pc_eclipse/build/CMakeFiles/run 
cd /home/neo/projects/lvgl/lv_port_pc_eclipse/build && /home/neo/projects/lvgl/lv_port_pc_eclipse/bin/main
ninja: build stopped: subcommand failed.

@kisvegabor
Copy link
Member Author

Do you use lv_ecample_lottie_1? I've seen a similar crash when the buffer wasn't large enough for the stride and alignment. (The example assumes stride==1 and align==4)

@XuNeo
Copy link
Collaborator

XuNeo commented Apr 19, 2024

Yes, I use lv_ecample_lottie_1 . I will try again later with stride == 1 && align == 4. Now they are both 64.

@FASTSHIFT
Copy link
Collaborator

@FASTSHIFT With the updated ThorVG the cogwheel has some artifacts here. Do you have any idea why could it happen?

OK, I'll try to investigate.

@XuNeo
Copy link
Collaborator

XuNeo commented Apr 20, 2024

I have figured it out:

diff --git a/examples/widgets/lottie/lv_example_lottie_1.c b/examples/widgets/lottie/lv_example_lottie_1.c
index 5e805f9f9..2d8cb4785 100644
--- a/examples/widgets/lottie/lv_example_lottie_1.c
+++ b/examples/widgets/lottie/lv_example_lottie_1.c
@@ -9,15 +9,16 @@ void lv_example_lottie_1(void)
 {
     extern const uint8_t lv_example_lottie_approve[];
     extern const size_t lv_example_lottie_approve_size;
-    static uint32_t buf[100 * 100];
     lv_obj_t * lottie = lv_lottie_create(lv_screen_active());
-    lv_lottie_set_buffer(lottie, 100, 100, buf);
+    lv_draw_buf_t * draw_buf = lv_draw_buf_create(100, 100, LV_COLOR_FORMAT_ARGB8888, LV_STRIDE_AUTO);
+    lv_lottie_set_draw_buf(lottie, draw_buf);
     //  lv_lottie_set_src_data(lottie, lv_example_lottie_approve, lv_example_lottie_approve_size);
     lv_obj_center(lottie);
 
     static uint32_t buf2[100 * 100];
+    lv_draw_buf_t * draw_buf2 = lv_draw_buf_create(100, 100, LV_COLOR_FORMAT_ARGB8888, LV_STRIDE_AUTO);
     lottie = lv_lottie_create(lv_screen_active());
-    lv_lottie_set_buffer(lottie, 100, 100, buf2);
+    lv_lottie_set_draw_buf(lottie, draw_buf2);
     lv_lottie_set_src_data(lottie, lv_example_lottie_approve, lv_example_lottie_approve_size);
 }
 
diff --git a/src/widgets/lottie/lv_lottie.c b/src/widgets/lottie/lv_lottie.c
index 024bf7edd..2f3d59017 100644
--- a/src/widgets/lottie/lv_lottie.c
+++ b/src/widgets/lottie/lv_lottie.c
@@ -85,7 +85,7 @@ void lv_lottie_set_draw_buf(lv_obj_t * obj, lv_draw_buf_t * draw_buf)
     }
 
     lv_lottie_t * lottie = (lv_lottie_t *)obj;
-    tvg_swcanvas_set_target(lottie->tvg_canvas, (void *)draw_buf->data, draw_buf->header.w, draw_buf->header.stride / 4,
+    tvg_swcanvas_set_target(lottie->tvg_canvas, (void *)draw_buf->data, draw_buf->header.stride / 4, draw_buf->header.w,
                             draw_buf->header.h, TVG_COLORSPACE_ARGB8888);
     tvg_canvas_push(lottie->tvg_canvas, lottie->tvg_paint);
     lv_canvas_set_draw_buf(obj, draw_buf);

@kisvegabor
Copy link
Member Author

I would like to use lv_lottie_set_buffer to statically allocate the large buffer and for simlicity.
I've updated the example, I think it's simple and works with non-default stride and alignment too.

XuNeo
XuNeo previously approved these changes Apr 21, 2024
FASTSHIFT
FASTSHIFT previously approved these changes Apr 22, 2024
@XuNeo
Copy link
Collaborator

XuNeo commented Apr 22, 2024

I would like to use lv_lottie_set_buffer to statically allocate the large buffer and for simlicity.

From #6101,
How about using LV_DRAW_BUF_DEFINE? Hardcode the size won't fit for GPUs.

static uint8_t buf[64 * 64 * 4 + LV_DRAW_BUF_ALIGN];

@hermet
Copy link

hermet commented Apr 22, 2024

@kisvegabor Hello, thank you for your efforts. I think Team LottieFiles could announce this news through its marketing channels. I'm wondering if you would like this idea and if you could provide a real working demo video (lottie animation on a real device with LVGL) us for the promotional post. I believe this is good for LVGL and Lottie Animation promotion! Thanks.

@kisvegabor kisvegabor dismissed stale reviews from FASTSHIFT and XuNeo via 313c129 April 22, 2024 18:45
@kisvegabor
Copy link
Member Author

@XuNeo

How about using LV_DRAW_BUF_DEFINE?

I was thinking about it too, but I found it a little bit obscure. I think in the example we should clearly show that it's just a simple array. However, if you wish I can add an example for draw_buf too and disable the current examples if the stride and align settings are not on default values.

@hermet

Hello, thank you for your efforts. I think Team LottieFiles could announce this news through its marketing channels.

That would be amazing! I'll share video when this PR is merged. Do you have a preferred vendor or board?

@FASTSHIFT Could you take look at the regression in image rendering?

@hermet
Copy link

hermet commented Apr 23, 2024

@kisvegabor I'm not considering any specific device in mind. I think a small device/screen would be good such as them: (if you can share the both sizes, it will be nice!)
image
image

Also, it would be helpful if we could have an approximate date for when we can share the demo video with the Team LottieFiles to push forward. Thanks!

Copy link
Collaborator

@liamHowatt liamHowatt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Super cool graphics format! Everything seems okay on Windows. Please update the Kconfig.

examples/widgets/lottie/lv_example_lottie.h Outdated Show resolved Hide resolved
examples/widgets/lottie/lv_example_lottie_2.c Outdated Show resolved Hide resolved
@kisvegabor
Copy link
Member Author

@hermet
Okay, I'll record a few videos showcasing Lottie on various devices. I've just checked I can compile ThorVG with Emscripten too, so LVGL+Lottie is working on web as well.

As soon as this rendering issue is resolved we can merge this PR. However in the meantime I can prepare the videos. I can record them on Thursday.
I'm planning grab some animations from LottieFiles but if you have some particular animation in mind, just let me know.

@kisvegabor
Copy link
Member Author

@hermet
I've just noticed something strange. If I set a larger size the animation is clipped. E.g. the example here works well in 64x64 size, but it looks like this in 200x200:
lottie
The clipped rectangle is 130x130 pixels.

Can it be a rounding issue of some kind in ThorVG?

@MichaelDu9226
Copy link

image
It seems that I also have a problem with clipped.

@hermet
Copy link

hermet commented Apr 24, 2024

@kisvegabor @MichaelDu9226 Hello, I'm a bit suspicious that the ThorVG canvas size is set correctly. Could you please double-check the what values(w, h, stride) did you set for the canvas size?

Result target(uint32_t* buffer, uint32_t stride, uint32_t w, uint32_t h, Colorspace cs) noexcept;  //C++
TVG_API Tvg_Result tvg_swcanvas_set_target(Tvg_Canvas* canvas, uint32_t* buffer, uint32_t stride, uint32_t w, uint32_t h, Tvg_Colorspace cs); //CAPI

Your plan is great. I suggest you the test-resources with these but you can choose any others as well.
lolo.json
lolo_walk.json

By the way, just for your information, when you encounter an issue with ThorVG, you can test the resource at www.thorvg.org/viewer. If it works properly there, then in most cases, the issue is likely on the user's side.

@kisvegabor
Copy link
Member Author

I'm a bit suspicious that the ThorVG canvas size is set correctly. Could you please double-check the what values(w, h, stride) did you set for the canvas size?

Yes, all there are set to 200. In www.thorvg.org/viewer all works well.

It's a little bit complicated to update ThorVG in LVGL so I'd try if it could be an issue in ThorVG.

@hermet
Copy link

hermet commented Apr 24, 2024

@kisvegabor To make it sure, so in this image, you set 200x200 tvg canvas. What is the actual content size on the screen? 130x130? or 200x200 with clipped?
image

@kisvegabor
Copy link
Member Author

Sorry, it was me. The lv_canvas size was on its default 130x130. It's working well now. Thank you for the quick replies 🙂

@kisvegabor
Copy link
Member Author

I've update the CI with 32 bit build option and now all passes. Please review.

tests/CMakeLists.txt Outdated Show resolved Hide resolved
XuNeo
XuNeo previously approved these changes May 22, 2024
@XuNeo
Copy link
Collaborator

XuNeo commented May 22, 2024

@W-Mai CI has reported the test_span asan error. I can reproduce it on my mac but not ubuntu. Please check if it's a real issue.

@kisvegabor
Copy link
Member Author

I can't reproduce it either on Linux. Interestingly it was passing here:
image

Rerunning just for out of curiosity. Do you see the ASAN error on Mac if you copy past the test code to a normal app?
Note these ALIGN and STRIDE settings of the tests.

@kisvegabor
Copy link
Member Author

kisvegabor commented May 22, 2024

It's passing on the second try (I've also squashed your fixes to my commit), however the span issue is shown in this PR too #6221.

It seems it's related to FreeType. My guess is the follwing:

  • As the span is being drawn with FreeType many draw_tasks are created for each letter.
  • We are using pthread, so the draw_tasks are drawn asynchronously.
  • Maybe the the draw_bufs are removed from FreeType's cache before they are actually drawn

So how does the cache of FreeType knows that a character is drawn (draw_buf is used) when releasing something from its cache. cc @W-Mai

This was referenced May 22, 2024
@XuNeo
Copy link
Collaborator

XuNeo commented May 22, 2024

Add #define LV_DRAW_SW_DRAW_UNIT_CNT 10 can expose more issues. Could you also take a look?

@W-Mai
Copy link
Contributor

W-Mai commented May 22, 2024

Add #define LV_DRAW_SW_DRAW_UNIT_CNT 10 can expose more issues. Could you also take a look?

Great!

It seems it's related to FreeType. My guess is the follwing:

As the span is being drawn with FreeType many draw_tasks are created for each letter.
We are using pthread, so the draw_tasks are drawn asynchronously.
Maybe the the draw_bufs are removed from FreeType's cache before they are actually drawn
So how does the cache of FreeType knows that a character is drawn (draw_buf is used) when releasing something from its cache. cc @W-Mai

You're right, I overlooked the impact of multithreading and forgot to it has already turn on the USE_OS configuration. 😅

@W-Mai
Copy link
Contributor

W-Mai commented May 22, 2024

I will try to fix this problem.

@kisvegabor
Copy link
Member Author

Thank you, @W-Mai!

@W-Mai
Copy link
Contributor

W-Mai commented May 22, 2024

@kisvegabor
Copy link
Member Author

Merging! 🎉🎉🎉

@kisvegabor kisvegabor merged commit f2e81d8 into lvgl:master May 23, 2024
17 checks passed
@kisvegabor kisvegabor deleted the feat/lottie branch May 23, 2024 16:25
XuNeo added a commit to XuNeo/thorvg that referenced this pull request May 24, 2024
Fix warning during build with lvgl lvgl/lvgl#6081

error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
Signed-off-by: Xu Xingliang <xuxingliang@xiaomi.com>
XuNeo added a commit to XuNeo/thorvg that referenced this pull request May 24, 2024
Fix warning during build with lvgl lvgl/lvgl#6081

error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
Signed-off-by: Xu Xingliang <xuxingliang@xiaomi.com>
XuNeo added a commit to XuNeo/thorvg that referenced this pull request May 24, 2024
Fix warning during build with lvgl lvgl/lvgl#6081

error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
Signed-off-by: Xu Xingliang <xuxingliang@xiaomi.com>
hermet pushed a commit to thorvg/thorvg that referenced this pull request May 27, 2024
Fix warning during build with lvgl lvgl/lvgl#6081

error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
Signed-off-by: Xu Xingliang <xuxingliang@xiaomi.com>
hermet pushed a commit to thorvg/thorvg that referenced this pull request May 31, 2024
Fix warning during build with lvgl lvgl/lvgl#6081

error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
Signed-off-by: Xu Xingliang <xuxingliang@xiaomi.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support Lottie animation
8 participants