Search FAQ

Gammon Forum

Notice: Any messages purporting to come from this site telling you that your password has expired, or that you need to verify your details, confirm your email, resolve issues, making threats, or asking for money, are spam. We do not email users with any such messages. If you have lost your password you can obtain a new one by using the password reset link.
 Entire forum ➜ MUSHclient ➜ Python ➜ Pickling
Pickling

Posting of new messages is disabled at present.

Refresh page

Pages: 1  2  3  4 

Posted by Shaun Biggs   USA  (644 posts)  Bio
Date Reply #45 on Wed 02 May 2007 01:46 PM (UTC)
Message
Quote:
Well it did what it was told to - moved the cursor to the start of the line twice, where the prompt overwrote it.

I understand, which is why I had correctly in quotes. It's just that for the first two, the text is displayed in the way that the respective OSes would normally display them, whereas the last one does not.

Quote:
In the case of Windows, I'm not sure, where it uses carriage-return/linefeed combinations, whether the carriage-return, or the linefeed, or the combination together, trigger the "logical end of line" processing.

The combination of the two is the "logical end of line" for Windows. Unicode gets around all of this by having U+0085 be "new line".
It is much easier to fight for one's ideals than to live up to them.
Top
Posted by David Haley   USA  (3,881 posts)  Bio
Date Reply #46 on Wed 02 May 2007 05:53 PM (UTC)
Message
What I don't understand about the Mac decision is that by only using \r, it is unclear what purpose \n is supposed to serve. At least in Unix, \n has a very clear purpose (end of line, start a new one) and \r has a very clear purpose (go to the beginning of the line). Windows does more or less the same thing. But in Mac, they've put \n functionality into \r, and left \n as useless, and as far as I know, there is no easy way to return to the beginning of the line.

In any case the real problem to me with the Mac's decision is the lack of integration; their GUI tools work in \r and the Unix environment under the hood works in \n.
David Haley aka Ksilyan
Head Programmer,
Legends of the Darkstone

http://david.the-haleys.org
Top
Posted by Shadowfyr   USA  (1,788 posts)  Bio
Date Reply #47 on Wed 02 May 2007 08:06 PM (UTC)

Amended on Wed 02 May 2007 08:07 PM (UTC) by Shadowfyr

Message
Hmm. Just going to throw one more thing in here. Under the old Apple IIs, the codes worked the same as the old machines. Clear screen, which on IBM systems worked by literally "clearing" the data, didn't exist on Apple IIs, so you used the "Form Feed" function. This worked much the same even *on* PCs if you where using ANSI based applications, like Telemate, to access a BBS. Form feed was "clear screen" and the ANSI command to clear the screen simply triggered a form feed, which dumped x lines to the screen, where x was 25 or 50, depending on the video mode. Later implementations broke this, and in the process created major confusion, since in something like a text box, form feed is "variable", so has to instead work as though it was clearing the data. This leads to the obvious confusion, such as a previous discussion here on text positioning functions and how they work, where no one is quite sure how the "clear screen" **should** work or how to handle that in a buffer.

In the case of telemate, it just inserted a form feed. As long as you didn't scroll "past" that "page" the output would just scroll normally. As soon as the form feed appeared in the text, designating a new page, and further scrolling up or down past the marker caused it to clear the output and redraw with the next page. Not knowing how to handle that causes serious issues in figuring out what the heck should be happening.

In any case. This is one of those cases where "technically" we are getting a conflict between what telnet, which was based on teletype systems in the first place, might be expected to do, and what compromises where made for things like storage and proper operation in file systems and document editors. And in this case we are running bang into a conflict because "technically" telnet *should* be handling the files as teletype, while the OS is mangling things into what the OS, file system and text editors expect to be dealing with. The two exist in literally different universes, and so apparently do all three OSes. lol
Top
Posted by Nick Gammon   Australia  (23,122 posts)  Bio   Forum Administrator
Date Reply #48 on Wed 02 May 2007 09:08 PM (UTC)
Message
Quote:

The combination of the two is the "logical end of line" for Windows.


Well, let's look at the keyboard for a moment. I take the phrase "logical end of line" to indicate "process the line I have typed" (or read from a file).

Now when I sit at a keyboard I don't hit two keys (carriage return, followed by linefeed) to make it process my command.

Interestingly, a small test indicates that when running under Linux (or Cygwin) pressing the Enter key is shown as being \n (linefeed), but under "Windows mode" the Enter key returns a carriage return. For example, in the MUSHclient source, you can see that in the code for the command window (sendvw.cpp) it detects VK_RETURN as the trigger for processing a "command" in the command window.

Then I tried to see what actually happens when you read a file in (eg. a .bat file read by the command processor).

I made 3 files using Ultra Edit in "hex editing" mode:

test1.bat : echo test1 \r\n echo test2 \r\n
test2.bat : echo test1 \n echo test2 \n
test3.bat : echo test1 \r echo test2 \r

Then using a command window I typed test1, test2, and test3 (to execute the bat files) to see which ones would echo the commands.

Both test1.bat and test2.bat worked, indicating that it is the \n on its own that terminates a "logical line" and the \r is really a filler, plus it is used to move the cursor to the start of the line under certain circumstances.

However test3 echoed "test1echo test2", which seemed to show that the command processing was triggered by the end-of-file condition, and indeed it did not even return the cursor to the start of the line.
- Nick Gammon

www.gammon.com.au, www.mushclient.com
Top
Posted by Nick Gammon   Australia  (23,122 posts)  Bio   Forum Administrator
Date Reply #49 on Wed 02 May 2007 10:00 PM (UTC)
Message
Just to make things perfectly clear, I have amended the documentation on this page:

http://www.gammon.com.au/scripts/doc.php?general=data_types


This makes it clear that strings cannot contain imbedded null characters.

This should make the documentation clearer, that in cases where you might be tempted to try to use hex 00 bytes (eg. in SendPkt) you will not be able to.
- Nick Gammon

www.gammon.com.au, www.mushclient.com
Top
Posted by Shadowfyr   USA  (1,788 posts)  Bio
Date Reply #50 on Fri 04 May 2007 12:55 AM (UTC)
Message
Umm. .bat is handled differently than say looking at the same file in something like DOS edit, and it may differ if ANSI.sys is loaded, which it isn't by default. Like I said, its a mess how this stuff actually gets dealt with on PCs, and it only gets odder when jumping OSes. The "hardware" can never generate anything other than 0x000D for the main key and 0x??0D for the keypad version. Most applications just strip off the first null/control and return the 0x0D. Linux apparently internally converts 0x0D from the hardware into 0x0A (If I remember the right code for LF). The hardware though is "still" producing the carriage return instead.
Top
Posted by David Haley   USA  (3,881 posts)  Bio
Date Reply #51 on Fri 04 May 2007 02:29 AM (UTC)
Message
Erm. This is the output from 'xev', a program that prints out every event that goes to it:

KeyPress event, serial 30, synthetic NO, window 0x4400001,
    root 0x1a5, subw 0x0, time 250593019, (-925,536), root:(466,946),
    state 0x10, keycode 36 (keysym 0xff0d, Return), same_screen YES,
"   XLookupString gives 1 bytes: (0d) "
"   XmbLookupString gives 1 bytes: (0d) "
    XFilterEvent returns: False

KeyRelease event, serial 33, synthetic NO, window 0x4400001,
    root 0x1a5, subw 0x0, time 250593107, (-925,536), root:(466,946),
    state 0x10, keycode 36 (keysym 0xff0d, Return), same_screen YES,
"   XLookupString gives 1 bytes: (0d) "
    XFilterEvent returns: False

KeyPress event, serial 33, synthetic NO, window 0x4400001,
    root 0x1a5, subw 0x0, time 250593443, (-925,536), root:(466,946),
    state 0x10, keycode 108 (keysym 0xff8d, KP_Enter), same_screen YES,
"   XLookupString gives 1 bytes: (0d) "
"   XmbLookupString gives 1 bytes: (0d) "
    XFilterEvent returns: False

KeyRelease event, serial 33, synthetic NO, window 0x4400001,
    root 0x1a5, subw 0x0, time 250593539, (-925,536), root:(466,946),
    state 0x10, keycode 108 (keysym 0xff8d, KP_Enter), same_screen YES,
"   XLookupString gives 1 bytes: (0d) "
    XFilterEvent returns: False


As you can see, the first press/release pair is for the normal enter, keysym 0xff0d, and the second pair is for the keypad, keysym 0xff8d.

Not sure where your ideas about what Linux is doing come from. Nor am I sure why you think the hardware is necessarily producing both cr/lf. The hardware is just sending in what key is pressed, not what a typewriter would do...
David Haley aka Ksilyan
Head Programmer,
Legends of the Darkstone

http://david.the-haleys.org
Top
Posted by Nick Gammon   Australia  (23,122 posts)  Bio   Forum Administrator
Date Reply #52 on Fri 04 May 2007 04:14 AM (UTC)
Message
Quote:

... but isn't the usage of replace() to remove something -hoping- it won't destroy another part of the string rather crude?


In an attempt to partly redress this problem, version 4.07 will now save tabs as 	 and then convert them when reading back in, thus tabs inside variables (and the "send" fields of triggers / aliases / timers) will now be preserved.
- Nick Gammon

www.gammon.com.au, www.mushclient.com
Top
Posted by Shaun Biggs   USA  (644 posts)  Bio
Date Reply #53 on Fri 04 May 2007 06:08 AM (UTC)

Amended on Fri 04 May 2007 06:10 AM (UTC) by Shaun Biggs

Message
Quote:
As you can see, the first press/release pair is for the normal enter, keysym 0xff0d, and the second pair is for the keypad, keysym 0xff8d.

The two keys do show up differently though. One identifies as "Return," and the other as "KP_Enter." Even the two control keys and the two alt keys have different codes. For proof, old versions of firefox didn't like it when I used the right control key to switch tabs. I had to use right control-tab. It's completely up to the software to interpret the hardware as you said. Also, when using the live CD for Gentoo and several other Linux distros, you need to select your keyboard type, which caused no end of frustration for a friend of mine who accidentally picked a German keyboard.
It is much easier to fight for one's ideals than to live up to them.
Top
Posted by David Haley   USA  (3,881 posts)  Bio
Date Reply #54 on Fri 04 May 2007 07:41 AM (UTC)
Message
Well, yes, they show up differently, but that's my point: they're different keys and it's just a keycode. Like you and I have said, it's not sending anything more than which key was pressed, the interpretation of which is up to the OS. I'm not sure where the claim that pressing return sends two hardware signals comes from.

That would, incidentally, mean that a Mac keyboard wouldn't work right on the PC, and a PC keyboard wouldn't work on a Mac either, and this is demonstrably false. It's kind of funny, actually, to see the "windows" key act as the "apple" key. :-)
David Haley aka Ksilyan
Head Programmer,
Legends of the Darkstone

http://david.the-haleys.org
Top
Posted by Shadowfyr   USA  (1,788 posts)  Bio
Date Reply #55 on Fri 04 May 2007 06:22 PM (UTC)
Message
I didn't say they produce CR+LF, I said that, according to what Nick was saying, Linux, or the application he was using, was translating it into LF, the same way that Windows apps tend to translate it into CR+LF. My point is that the **hardware** will still generate the "same" code no matter what the software insists it turns into.

And, just to be clear **modern** keyboards between MAC and PC are equivalent. The older model of MACs had an unusual feature. While most software couldn't tell the difference between keypresses from multiple keyboards, you could *chain* them and if your application requested the unique ID as part of the key code, you could have up to like 32 (64?) keyboards, or something, each generating its own data, all identifiable as separate devices. In some ways the shift to PC style hardware undermined some of their own innovations. Not that a lot of people used them that way, but you could, in theory, chain any number of different input devices through the connector, not just keyboards. Sadly, I don't think anyone ever really figured out what the hell to do with that feature. lol
Top
Posted by Isthiriel   (113 posts)  Bio
Date Reply #56 on Fri 04 May 2007 07:39 PM (UTC)
Message
Wikipedia's entry on Newlines (and specifically the section on History and Newlines-in-programming) is helpful.

Pressing the enter key generates a mechanical or electrical signal that is interpreted by the keyboard's controller and translated to PS/2 keycodes (this is important for keys that share part of their mask(?) like on the g15 the right-shift key is ignored if you are already holding the cursor-up and cursor-right keys down) which are then forwarded to the motherboard's BIOS and from there as an interrupt to software... not necessarily the OS.

"modern" Mac and PC keyboards are interoperable because they are both conformant to the PS/2 standard (mostly because of economies of scale). The Windows/Mac Command key being identified as equivalent /could/ be because they use the same keycode, but it is just as likely to be OS X recognizing the windows key in software.

(As a side note I can chain up to 127 G15s together... but unless you have software that can identify which keyboard the event came from they are treated identically by Windows.)

Applications have the option to handle keycodes themselves but most rely on the operating system for the low level stuff (for.ex on Linux, assuming stdin has something to do with a keyboard...), and that's where libc bites you. Files are binary data and newlines have a concrete representation on disk but in memory, provided it acts like a newline your implementation of libc decides how it is actually represented. So fread and fwrite do a conversion if the file was opened in text mode ('r' and 'w') instead of binary mode ('rb' and 'wb'). It's a nuisance. (Amongst other reasons it tends to make ftell useless.)
Top
Posted by Shadowfyr   USA  (1,788 posts)  Bio
Date Reply #57 on Sat 05 May 2007 10:14 PM (UTC)
Message
Interesting.. I wasn't aware that you "could" tell keyboards apart under the PS/2 architecture or that they made any that could even be chained for that... Explains I supposed why I could never figure out why the Cue Cat gadget didn't generate keycodes in software without a driver (or modification of the Cue Cat/both??). In theory, if it was simply sending the same keycodes to the hardware, any software *should* have seen them, encoded with their screwy encryption or not.

Which reminds me, I need to figure out how to declaw mine at some point. One of these days I need a better system to catalog my books and DVDs, which scanning the barcode would make a lot easier. lol
Top

The dates and times for posts above are shown in Universal Co-ordinated Time (UTC).

To show them in your local time you can join the forum, and then set the 'time correction' field in your profile to the number of hours difference between your location and UTC time.

149,223 views.

This is page 4, subject is 4 pages long:  [Previous page]  1  2  3  4 

Posting of new messages is disabled at present.

Refresh page

Go to topic:           Search the forum

[Go to top] top

Information and images on this site are licensed under the Creative Commons Attribution 3.0 Australia License unless stated otherwise.

Home