Opened 4 years ago

Closed 3 years ago

#43 closed defect (fixed)

unread-char doesn't change file-position

Reported by: rtoy Owned by:
Priority: major Milestone:
Component: Unicode Version: 20b
Keywords: Cc:

Description

This test is from Douglas Crosher, on the maxima mailing list.

(with-open-file (ostream "ctest.txt" :direction :output
             :external-format #+clisp "utf-8" #-clisp :utf-8)
  (dotimes (i 1000)
    (write-char (code-char #x1234) ostream)))

(with-open-file (stream "ctest.txt" :direction :input
            :external-format #+clisp "utf-8" #-clisp :utf-8)
  (let ((p0 (file-position stream))
    (ch (read-char stream)))
    (unread-char ch stream)
    (let ((p0* (file-position stream)))
      (if (eql p0* p0) "Ok" "Broken"))))

Cmucl returns "Broken" because p0 = 0 but p0* = 2. I think unread-char didn't update everything needed by our unicode stream buffers.

Change History (3)

comment:1 Changed 4 years ago by rtoy

  • Resolution set to fixed
  • Status changed from new to closed

Fixed and should be available in the April snapshot.

The issue was caused by FAST-READ-CHAR-STRING-REFILL not updating the ibuf head pointer when some octets in the buffer were not converted to characters because the last octets in the buffer do not form a complete character.

comment:2 Changed 3 years ago by rtoy

  • Resolution fixed deleted
  • Status changed from closed to reopened

The fix that was implemented in stream.lisp does fix this issue. However, it causes the Unicode tests in src/i18n/tests to fail. Cmucl ends up reading in the whole file into one string and causes an error.

The change has been reverted, pending a better fix. This ticket is then, of course, reopened.

comment:3 Changed 3 years ago by rtoy

  • Resolution set to fixed
  • Status changed from reopened to closed

This has been fixed (again) in a different way. The fix is in fd-stream.lisp in FILE-POSITION, which needs to account for any unprocessed characters as well as any octets that are in the buffer but have not been converted to characters yet.

Note: See TracTickets for help on using tickets.