quicklisp 2018-10-18

Test diff with the previous release:
https://common-lisp.net/project/cl-test-grid/ql/quicklisp-2018-10-18-diff2.html

quicklisp 2016-09-29

Test results for quicklisp 2016-09-29 comparing to the previous release are coming here.

If you need help reproducing or understanding particular failure, email or comment.

quicklisp 2016-05-31

The difference between this and previous months:

Grouped by lisp implementation first and then by library:

https://common-lisp.net/project/cl-test-grid/ql/quicklisp-2016-05-31-diff.html

Grouped by library first and then by lisp impl:

https://common-lisp.net/project/cl-test-grid/ql/quicklisp-2016-05-31-diff2.html

(Both reports show the same data, just arranged differently)

This time the diff is smaller than the past month. There are some regressions, some improvements.

If you're interested in some particular failure or need help investigating something, just ask in the comments or on the mailing list.

quicklisp 2016-04-21

The difference between this and previous months:

Grouped by lisp implementation first and then by library:

https://common-lisp.net/project/cl-test-grid//ql/quicklisp-2016-04-21-diff.html

Grouped by library first and then by lisp impl:

https://common-lisp.net/project/cl-test-grid//ql/quicklisp-2016-04-21-diff2.html

(Both reports show the same data, just arranged differently)

As usually, some new libraries start to fail on old lisp implementations because they need newer ASDF.

3d-vectors redefines constant to a value not eql to the previous value. clinch starts to fail on several lisps. hyperluminal-mem refers undefined variable MOST-POSITIVE-B+SIZE. rutils crashes CCL. And some other failures. There are improvements too of course.

If you're interested in some particular failure or need help investigating something, just ask in the comments or on the mailing list.

quicklisp 2016-02-08

The difference between this and previous months:

Grouped by lisp implementation first and then by library:

https://common-lisp.net/project/cl-test-grid/ql/quicklisp-2016-02-08-diff.html

Grouped by library first and then by lisp impl:

https://common-lisp.net/project/cl-test-grid/ql/quicklisp-2016-02-08-diff2.html

(Both reports show the same data, just arranged differently)

The diff contains mostly improvements, but there are several regressions: varjo, gendl, lake, buffalo, binfix, data-table and others.

If you're interested in some particular failure or need help investigating something, just ask in the comments or on the mailing list.

quicklisp 2015-12-18

Almost every month cl-test-grid detects various regressions in CL ecosystem.

Like this month:

https://common-lisp.net/project/cl-test-grid/ql/quicklisp-2015-12-18-diff2.html

And in general: https://common-lisp.net/project/cl-test-grid/ql/

I have problem reporting all these regressions to interested parties. Studying why something fails, searching for right person to report, formulating and submitting bug reports - all that easily takes more than a full working day.

Also, sometimes it's not necessary a bug from the library maintainer point of view, for example if his library fails on CLISP because CLISP ships with old ASDF, maybe this maintainer doesn't care. But maybe those who depends on this library care. So, it's not always obvious whether the situation requires to disturb people.

That's why I created this blog. I mean it as a low volume notification channel - mostly a notice with reports once a month when the tests are finished, and a summary of what I understand about the failures. Then, if people are interested in particular failures, they can ask and I will help to investigate and explain why exactly something fails.

I hope this will help to effectively fix important failures and keep the community aware about the factors causing fluctuations of quality in CL ecosystem.

Now about this month regressions.

First of all, I usually build two diff reports.

One is grouped by lisp implementation first and then by library:

https://common-lisp.net/project/cl-test-grid/ql/quicklisp-2015-12-18-diff.html

Another one is grouped by library first and then by lisp impl:

https://common-lisp.net/project/cl-test-grid/ql/quicklisp-2015-12-18-diff2.html

Both reports show the same data, just arranged differently.

The most noticeable regressions in quicklisp 2015-12-18 comparing to the previous

release:

• bordeaux-threads uses UIOP without specifying :depends-on :uiop in the .asd file; in result on lisp implementations where ASDF doesn't include UIOP, bordeaux-threads (and libraries depending on it) fail to load
• iolib rejects ASDF < 3.1, so fails to load (and prevents libraries using it from loading) on lisps with elder ASDF
• cl+ssl declares FFI binding for function SSL_CTX_set_default_verify_dir which is absent in the latest stable OpenSSL release, and only present in OpenSSL master. c+ssl doesn't use this function, the ffi declaration was added just to allow others to invoke it. Still, cl+ssl fails to load on CMUCL because CMUCL actually checks for foreign symbol to be present when FFI declaration is processed. Other lisps we tested are unaffected. This problem is fixed already in cl+ssl git.
• As usually, more and more libraries start to use new ASDF features, for example keywords :home, :at. These libraries and their dependencies fail to load on elder ASDF.
• croatoan library starts using sb-ext:posix-getenv thus fails on non-sbcl lisps (hint: use uiop:getenv instead)
• rutils (and depending libs) fail on ABCL and ECL with "No such method for PRINT-OBJECT"
• several libraries fail on ECL with "* is not a valid type specifier"
• com.informatimago.common-lisp.telnet fails on CMUCL with "The value of the :SLOT-NAME initarg, US, is a constant and so cannot be bound."

... and other failures.

If you're interested in some specific failure or need help investigating something, just ask in the comments or on the mailing list.

Regressions and improvements in quicklisp 2015-08-04

The difference between quicklisp 2015-07-10 and quicklisp 2015-08-04 is shown in these reports:

https://common-lisp.net/project/cl-test-grid/ql/quicklisp-2015-08-04-diff.html
grouped by lisp implementation

https://common-lisp.net/project/cl-test-grid/ql/quicklisp-2015-08-04-diff2.html
grouped by library

(Both reports show the same data, just arranged differently)

As we see, most of the new failures are caused by drakma failing to load on CCL 1.8, CLISP and SBCL 1.0.58, because drakma.asd now specifies test-op, but in a way supported only by new ASDF, and not supported by the ASDF shipped with these old lisp implementations. Maybe the drakma-caused failures are not critical for everyone, on the other hand it is trivial to fix.

There are other regressions as well as some improvements. If you care about some of the results, but don't understand the cause of particular failure, or how to read the report, just ask in the comments or on the mailing list, we will be glad to help, investigate and reproduce the problem.

BTW, you can click test result links in the reports to open the corresponding log.

A little bit more about the cl-test-grid project. We regularly collect test results of all the Quicklisp libraries on various lisp implementations, and compute difference between Quicklisp versions to detect changes (regressions or improvements). That way we can monitor the CL ecosystem as a whole.