Opened 3 years ago
Closed 3 years ago
#61912 closed defect (fixed)
opencv @3.4.13: only 17% of test suite passes
| Reported by: | kencu (Ken) | Owned by: | Christopher Nielsen <mascguy@…> |
|---|---|---|---|
| Priority: | Normal | Milestone: | |
| Component: | ports | Version: | |
| Keywords: | Cc: | mascguy (Christopher Nielsen) | |
| Port: | opencv |
Description
Quite pathetic performance on opencv's test suite.
Running on Catalina, Xcode 12.3, only 17% of tests pass. I'm not sure at this moment what portion of this might relate to the MacPorts environment, and what part of it is just opencv itself.
Anyone try to actually use opencv for anything on MacOS should be aware of this issue.
To enable testing:
% git diff
diff --git a/graphics/opencv/Portfile b/graphics/opencv/Portfile
index 936a6c28422..da4afa75993 100644
--- a/graphics/opencv/Portfile
+++ b/graphics/opencv/Portfile
@@ -147,8 +147,8 @@ configure.args-append \
-DBUILD_DOCS=OFF \
-DBUILD_EXAMPLES=OFF \
-DBUILD_PACKAGE=OFF \
- -DBUILD_PERF_TESTS=OFF \
- -DBUILD_TESTS=OFF \
+ -DBUILD_PERF_TESTS=ON \
+ -DBUILD_TESTS=ON \
-DBUILD_WITH_DYNAMIC_IPP=OFF \
\
-DBUILD_ZLIB=OFF \
@@ -514,6 +514,9 @@ post-destroot {
}
}
+test.run yes
+test.target test
+
livecheck.type regex
livecheck.url https://opencv.org/releases/
livecheck.regex {archive/(3\.[0-9.]+)?\.[tz]}
results
17% tests passed, 34 tests failed out of 41 Label Time Summary: Accuracy = 357.03 sec*proc (17 tests) Main = 1315.69 sec*proc (41 tests) Performance = 875.31 sec*proc (12 tests) Sanity = 83.35 sec*proc (12 tests) opencv_calib3d = 98.62 sec*proc (3 tests) opencv_core = 427.26 sec*proc (3 tests) opencv_dnn = 26.62 sec*proc (3 tests) opencv_features2d = 62.15 sec*proc (3 tests) opencv_flann = 0.50 sec*proc (1 test) opencv_highgui = 0.84 sec*proc (1 test) opencv_imgcodecs = 23.89 sec*proc (3 tests) opencv_imgproc = 530.95 sec*proc (3 tests) opencv_ml = 3.44 sec*proc (1 test) opencv_objdetect = 57.65 sec*proc (3 tests) opencv_photo = 0.63 sec*proc (3 tests) opencv_shape = 0.31 sec*proc (1 test) opencv_stitching = 0.60 sec*proc (3 tests) opencv_superres = 12.64 sec*proc (3 tests) opencv_video = 1.54 sec*proc (3 tests) opencv_videoio = 64.17 sec*proc (3 tests) opencv_videostab = 3.89 sec*proc (1 test) Total Test time (real) = 1315.97 sec The following tests FAILED: 1 - opencv_test_core (Failed) 3 - opencv_sanity_core (Failed) 5 - opencv_test_imgproc (Failed) 6 - opencv_perf_imgproc (Failed) 7 - opencv_sanity_imgproc (Failed) 8 - opencv_test_ml (Failed) 9 - opencv_test_photo (Failed) 10 - opencv_perf_photo (Failed) 11 - opencv_sanity_photo (Failed) 12 - opencv_test_video (Failed) 13 - opencv_perf_video (Failed) 14 - opencv_sanity_video (Failed) 15 - opencv_test_dnn (Failed) 16 - opencv_perf_dnn (Failed) 17 - opencv_sanity_dnn (Failed) 18 - opencv_test_features2d (Failed) 19 - opencv_perf_features2d (Failed) 20 - opencv_sanity_features2d (Failed) 21 - opencv_test_imgcodecs (Failed) 24 - opencv_test_shape (Failed) 25 - opencv_test_videoio (Failed) 26 - opencv_perf_videoio (Failed) 27 - opencv_sanity_videoio (Failed) 28 - opencv_test_calib3d (Failed) 29 - opencv_perf_calib3d (Failed) 30 - opencv_sanity_calib3d (Failed) 32 - opencv_test_objdetect (Failed) 33 - opencv_perf_objdetect (Failed) 34 - opencv_sanity_objdetect (Failed) 35 - opencv_test_stitching (Failed) 36 - opencv_perf_stitching (Failed) 37 - opencv_sanity_stitching (Failed) 38 - opencv_test_superres (Failed) 40 - opencv_sanity_superres (Failed) Errors while running CTest
Change History (5)
comment:1 Changed 3 years ago by mascguy (Christopher Nielsen)
| Cc: | mascguy added |
|---|
comment:2 Changed 3 years ago by mascguy (Christopher Nielsen)
comment:3 Changed 3 years ago by kencu (Ken)
Well, turns out opencv does not do a whole lot better on Ubuntu 20.10 either:
24% tests passed, 31 tests failed out of 41 Label Time Summary: Accuracy = 614.58 sec*proc (17 tests) Main = 8285.79 sec*proc (41 tests) Performance = 6942.99 sec*proc (12 tests) Sanity = 728.22 sec*proc (12 tests) opencv_calib3d = 688.18 sec*proc (3 tests) opencv_core = 3091.53 sec*proc (3 tests) opencv_dnn = 110.29 sec*proc (3 tests) opencv_features2d = 294.62 sec*proc (3 tests) opencv_flann = 0.07 sec*proc (1 test) opencv_highgui = 1.21 sec*proc (1 test) opencv_imgcodecs = 39.52 sec*proc (3 tests) opencv_imgproc = 3693.58 sec*proc (3 tests) opencv_ml = 6.73 sec*proc (1 test) opencv_objdetect = 124.57 sec*proc (3 tests) opencv_photo = 0.91 sec*proc (3 tests) opencv_shape = 0.16 sec*proc (1 test) opencv_stitching = 91.91 sec*proc (3 tests) opencv_superres = 126.13 sec*proc (3 tests) opencv_video = 2.53 sec*proc (3 tests) opencv_videoio = 6.24 sec*proc (3 tests) opencv_videostab = 7.62 sec*proc (1 test) Total Test time (real) = 8286.56 sec The following tests FAILED: 1 - opencv_test_core (Failed) 3 - opencv_sanity_core (Failed) 5 - opencv_test_imgproc (Failed) 6 - opencv_perf_imgproc (Failed) 7 - opencv_sanity_imgproc (Failed) 8 - opencv_test_ml (Failed) 9 - opencv_test_photo (Failed) 10 - opencv_perf_photo (Failed) 11 - opencv_sanity_photo (Failed) 12 - opencv_test_video (Failed) 13 - opencv_perf_video (Failed) 14 - opencv_sanity_video (Failed) 15 - opencv_test_dnn (Failed) 16 - opencv_perf_dnn (Failed) 17 - opencv_sanity_dnn (Failed) 18 - opencv_test_features2d (Failed) 19 - opencv_perf_features2d (Failed) 20 - opencv_sanity_features2d (Failed) 21 - opencv_test_imgcodecs (Failed) 24 - opencv_test_shape (Failed) 25 - opencv_test_videoio (Failed) 28 - opencv_test_calib3d (Failed) 29 - opencv_perf_calib3d (Failed) 30 - opencv_sanity_calib3d (Failed) 32 - opencv_test_objdetect (Failed) 33 - opencv_perf_objdetect (Failed) 34 - opencv_sanity_objdetect (Failed) 35 - opencv_test_stitching (Failed) 36 - opencv_perf_stitching (Failed) 37 - opencv_sanity_stitching (Failed) 40 - opencv_sanity_superres (Failed) Errors while running CTest make: *** [Makefile:130: test] Error 8
so perhaps the test suite needs some work for this software...I have not as yet tried the 4.x releases.
comment:4 Changed 3 years ago by mascguy (Christopher Nielsen)
Optional variant +tests added via pull request, for both opencv and opencv4:
comment:5 Changed 3 years ago by Christopher Nielsen <mascguy@…>
| Owner: | set to Christopher Nielsen <mascguy@…> |
|---|---|
| Resolution: | → fixed |
| Status: | new → closed |
Note: See
TracTickets for help on using
tickets.
Ken, I'll add a
testsvariant, for bothopencvandopencv4. Thanks!