Ticket #43 (closed defect: fixed)

Opened 3 years ago

Last modified 3 years ago

unread-char doesn't change file-position

Reported by: rtoy Owned by:
Priority: major Milestone:
Component: Unicode Version: 20b
Keywords: Cc:


This test is from Douglas Crosher, on the maxima mailing list.

(with-open-file (ostream "ctest.txt" :direction :output
             :external-format #+clisp "utf-8" #-clisp :utf-8)
  (dotimes (i 1000)
    (write-char (code-char #x1234) ostream)))

(with-open-file (stream "ctest.txt" :direction :input
            :external-format #+clisp "utf-8" #-clisp :utf-8)
  (let ((p0 (file-position stream))
    (ch (read-char stream)))
    (unread-char ch stream)
    (let ((p0* (file-position stream)))
      (if (eql p0* p0) "Ok" "Broken"))))

Cmucl returns "Broken" because p0 = 0 but p0* = 2. I think unread-char didn't update everything needed by our unicode stream buffers.

Change History

Changed 3 years ago by rtoy

  • status changed from new to closed
  • resolution set to fixed

Fixed and should be available in the April snapshot.

The issue was caused by FAST-READ-CHAR-STRING-REFILL not updating the ibuf head pointer when some octets in the buffer were not converted to characters because the last octets in the buffer do not form a complete character.

Changed 3 years ago by rtoy

  • status changed from closed to reopened
  • resolution fixed deleted

The fix that was implemented in stream.lisp does fix this issue. However, it causes the Unicode tests in src/i18n/tests to fail. Cmucl ends up reading in the whole file into one string and causes an error.

The change has been reverted, pending a better fix. This ticket is then, of course, reopened.

Changed 3 years ago by rtoy

  • status changed from reopened to closed
  • resolution set to fixed

This has been fixed (again) in a different way. The fix is in fd-stream.lisp in FILE-POSITION, which needs to account for any unprocessed characters as well as any octets that are in the buffer but have not been converted to characters yet.

Note: See TracTickets for help on using tickets.