read-sequence vs unicode

[Raymond Toy]

Cmucl has been able to use READ-SEQUENCE to read octets (and other integers types) from character streams. With the introduction of Unicode support, this no longer works correctly in general. The data that is read is not done from the last position, and the data that is read is not necessarily reflected in the next READ-CHAR. That is, READ-CHAR might re-read the data that READ-SEQUENCE already read. (This depends on how much data has been read, and the internal stream buffering.)

However, if the external format is :iso8859-1, then READ-SEQUENCE behaves as it used to. Hence, as a workaround, the user can set the external format to :iso8859-1 before READ-SEQUENCE and set it back afterwards. This works as expected.

Perhaps READ-SEQUENCE should do that itself? (Appropriately wrapping everything in UNWIND-PROTECT so that the stream external format isn't unexpectedly modified.)

[Raymond Toy]

The October snapshot should have this mostly fixed. Errors will now be signaled if the stream is of the wrong type, but if the stream is a binary-text-stream, READ-SEQUENCE will work as before. This is a change from 20a and 20b, and is also incompatible with 19f and earlier which allowed READ-SEQUENCE to work on character streams and binary streams of '(unsigned-byte 8).

[Raymond Toy]

Closing report. It seems to be working as expected---errors are signaled if the stream is of the wrong type, except if the stream is a binary-text-stream.