Wikipedia:Reference desk/Archives/Computing/2010 August 18

Welcome to the Wikipedia Computing Reference Desk Archives
The page you are currently viewing is an archive page. While you can leave answers for any questions shown below, please ask new questions on one of the current reference desk pages.

August 18

Surprising SEGV from read(2)

Resolved

Under what circumstances can this code cause a segmentation fault in read()?

#include<unistd.h>
#include<string.h>
int cread(int fd,void *buf,int sz) {
  memset(buf,0,sz);
  return read(fd,buf,sz);
}

Because the prototypes are there, sz will be properly promoted to a size_t. What does read() do (if, say, other memory is corrupted) that could cause it to fail? --Tardis (talk) 00:23, 18 August 2010 (UTC)

Well, trivially if fd is not a valid file descriptor (depending on the quality of your libc, of course). Can you verify if fd is valid? --Stephan Schulz (talk) 00:31, 18 August 2010 (UTC)

You're supposed to be able to call read(2) with an invalid descriptor and just get EBADF. fd==20 here, for what that's worth. --Tardis (talk) 01:06, 18 August 2010 (UTC)

If you're seeing this, I'd suspect that read is doing some pointer arithmetic that memset isn't, and that a vastly wrong value for sz is causing it to overflow or underflow, or perhaps memset has some internal check that's causing it to silently stop before it faults, a check that isn't done in read. I tried some trivially mad values for sz (-1, 0x7fffffff, 0x80000001) and it's memset that faults for me, but your libc, or your sz, may differ. -- Finlay McWalter • Talk 00:50, 18 August 2010 (UTC)

The thing that gets me is that read is a system call! The kernel needs no help from me to put bytes into that buffer, and I've already demonstrated that the buffer is legit. What pointer arithmetic could it possibly need to do? --Tardis (talk) 01:06, 18 August 2010 (UTC)

You haven't really tested that the buffer is legit - you're assuming that memset will have addressed all that memory, because that's what is contract appears to say it will. But what does memset really do when you give it a vastly, meaninglessly negative sz? Isn't it within is right to do nothing at all? If you really want to test the buffer is legit, you'll write the memory yourself with a loop. -- Finlay McWalter • Talk 01:15, 18 August 2010 (UTC)

True enough: its argument is unsigned, but it could in theory ignore very large values. But I also know that sz==67480 (and that the buffer is actually 512000 B long). --Tardis (talk) 01:51, 18 August 2010 (UTC)

Oh, and if buf is on the stack, and you trash the stack with an sz that's too big (or -ve), you can segfault in several ways - you can trash cread's stack frame, meaning it'll return to 0 (bang!) or (possibly with a -ve sz) mangle cread's copy of the pointer called buf, so that it no longer points to the actual buffer on the stack, but instead to 0 (bang!). That shouldn't be the case if buf is malloced, or is in .bss or .data, however. If that's the case, commenting out the memset should cause the read to succeed. -- Finlay McWalter • Talk 01:07, 18 August 2010 (UTC)

To clarify, I'm suggesting that memset is trashing the stack, but that you don't notice until the read. -- Finlay McWalter • Talk 01:09, 18 August 2010 (UTC)

buf is obtained from malloc(), but I think you're right about the stack; attaching a debugger shows that the pointer that triggers the SEGV points 440 MB above the stack pointer, and is itself on the stack — but then the SEGV shows up in a different function. When I let it die and look at the core, it says it died inside read() and with a completely different pointer. --Tardis (talk) 01:51, 18 August 2010 (UTC)

Ah, you're screwed then:) If the stack is corrupt, and you can't use tools to help, you're reduced to putting in canaries into the stack (just declaring autos with known odd values like 0x3f5c) and periodically checking them to see when they're intact and when they've been steamrollered. -- Finlay McWalter • Talk 02:09, 18 August 2010 (UTC)

When I've had to do that in the past, I wrote a little library with register_canary(addr, val) and unregister_canary(addr) which stored a little database of canaries. Another thread woke up every 1ms or so and verified all the canaries were intact. If one is missing it segfaults into the debugger deliberately. -- Finlay McWalter • Talk 02:14, 18 August 2010 (UTC)

In general, this is perfect fodder for valgrind or purify; you're seeing a segfault in read because that's where memory misbehaviour was detected, but if memory is already corrupted by some other bad code elsewhere, you'll never find the corruption by worrying about what read does. -- Finlay McWalter • Talk 01:19, 18 August 2010 (UTC)

Unfortunately, this is running under MPI, and is (of course) only failing in parallel; running it with such tools is rather more difficult than it would be otherwise. Thus my interest in a theoretical analysis that might point me at the right part of the code. --Tardis (talk) 01:51, 18 August 2010 (UTC)

Here's why you can't rely on one function accessing memory identically to another (particularly when passed pathological parmeters). Consider the following two (very naive) implementations:

  void simple_memset(char* p, byte n, int sz) { 
    char * d = p;
    while (d < (p+count)){
      *d=n;
      d++;
    }
  }

  void simple_read(int fd, char * p, int sz) {
    int bytes_read=0;
    while(bytes_read<sz){
      p[bytes_read++]=get_byte_from_file(fd);
    }
  }

Those both look like reasonable implementations. Now consider the following example, assuming a 16 bit address space (I'm too lazy to type all those extra 0000s, but the point is the same in 32 or 64 bits). Say your data segment is located at A000..BFFF, with buf 0x100 bytes beginning at B000. Now say you mess up and instead of passing 0x100 as sz you pass 0x7654. Inside simple_memset, at the beginning, d is 0xB000, p is 0xB000, and count is 0x7654. So p+count would sum to 0x12654, but that just truncates to 0x2654. As d >= 0x2654, simple_memset will terminate without writing any bytes. So that call to simple_memset hasn't validated you can access the buffer. And lo, look what happens when you then run simple_read. It works for a while, even reading far off the end buf buf at 0xB0FF, but eventually bytes_read gets to 0x1000 (which is allowed, because that's much less than 0x7654). It does that p[bytes_read++], where p is 0xB000 and bytes_read is 0x1000, so it's dereferencing memory at 0xC000, which is beyond the bounds of the data segment, and that's a segfault. -- Finlay McWalter • Talk 01:59, 18 August 2010 (UTC)

It was in fact stack-smashing, produced by a truly remarkably bad set of communication functions that sent the wrong data and then stored that incorrect data into (rather than through!) a pointer. I wish I could say that the code was ancient and written by some idiot long since departed, but in fact I wrote it on the 4th of this month, so… yeah. Thanks for reminding me of the obvious. --Tardis (talk) 02:32, 18 August 2010 (UTC)

Worse, that should generally generate a warning, and some idiot ignored that warning and thought "ah, I'll fix that later" :) -- Finlay McWalter • Talk 02:36, 18 August 2010 (UTC)

Unfortunately, generic interfaces like MPI offer no such type safety:

void recv3(int *p,int src) {
  MPI_Status st;
  MPI_Recv(p,1,MPI_INT,src,0,MPI_COMM_WORLD,&st);  /* convert int* to void*: OK */
  MPI_Recv(&p,1,MPI_INT,src,0,MPI_COMM_WORLD,&st); /* convert int** to void*: OK?! */
  MPI_Recv(*p,1,MPI_INT,src,0,MPI_COMM_WORLD,&st); /* convert int to void*: warning */
}

I may be that idiot, but I run gcc with -pedantic -Wall -Wextra -Wfloat-equal -Wundef -Wredundant-decls -Wpointer-arith -Wwrite-strings -Wshadow -Winline -Wdisabled-optimization -Wstrict-prototypes -Wunreachable-code. --Tardis (talk) 14:24, 18 August 2010 (UTC)

Lint (software) can often catch pointer and cast conversions that the -pedantic warnings do not catch... this chapter from Linux Clusters discusses the use of splint with MPI; I have never used that tool, but it looks like it can check for common argument mismatches in MPI functions. Nimur (talk) 20:33, 18 August 2010 (UTC)

wget

Would it be possible for wget to scan say 5 pages and then output a list of every link on those pages into a text file? 82.44.54.4 (talk) —Preceding undated comment added 11:35, 18 August 2010 (UTC).

Uhh, as far as I know, you can't do that using wget by itself. You could use wget to download the files you want, then use sed to process those files and filter for the <a></a> HTML tags. CaptainVindaloo ^{t c e} 19:07, 18 August 2010 (UTC)

It depends on what you need the URLs for. If you want to create a text file just to feed it back to wget at a later time, extracting the URLs is unnecessary. Just specify the downloaded page as input file and omit the URL(s) on the command line. See the help file for commandline options -i, -F (and you might need -B as well). -- 78.43.71.155 (talk) 20:31, 18 August 2010 (UTC) PS: Prithee, do tell: What are you up to? Creating a local copy of 4chan?

Mutation in Genetic Algorithms (optimization)

In the Mutation(GA) article, it is not mentioned where the mutation operation is used in the GA. Let's say there are N chromosomes in the last step, "old N chromosomes". I think there are three choises to create the new population.

1) N new chromosomes are generated from three different operations: a) Some are directly copied from initial population b) Some are generated by crossover c) Some are generated by mutation (This is the algoritm used in MATLAB's implementation)

2) N old chromosomes enter crossover, after mating and crossover N new chromosomes are generated. N new chromosomes enter mutation. Best N chromosomes are selected out of 2N chromosomes.

3) N old chromosomes enter crossover, after mating and crossover N new chromosomes are generated. Both N old and N new chromosomes enter mutation. Best N chromosomes are selected out of 2N chromosomes.

Which one above is true? OR can I use any of them? Kavas (talk) 12:19, 18 August 2010 (UTC)

I could be wrong, but I suspect you'll probably be better off asking this on the Wikipedia:Reference_desk/Science reference desk, this is computing and this doesn't (to me) seem related?  ZX81  ^talk 18:50, 18 August 2010 (UTC)

cf Wikipedia:Reference desk/Science#Mutation in genetic algorihms, yesterday. -- Finlay McWalter ☻ Talk 20:08, 18 August 2010 (UTC)

I asked that question too. But, as I use a numerical computing environment (MATLAB) for implementing the GAs, I thought "Computing Desk" should be more suitable. I'm not sure "mutational meltdown" refers to "stuck into a local minimum" there. Kavas (talk) 21:49, 18 August 2010 (UTC)

Reinstalled OSes, regedit, and Star Wars

I have Star Wars: Empire at War, and its expansion, both legally bought and paid for. I installed them on my computer. Then my OS (Windows XP Home) was eaten by viruses. I also had Windows XP Professional on the computer because I was aware that this might happen. I am now running XP Professional. The problem lies in that I still have Empire at War and Forces of Corruption installed, but not listed in the registry. Can you please tell me what registry keys are necessary for the game to run, and what their contents are? I would reinstall from disk, but I think my EAW disk 1 might be corrupted (it won't run the installation screen even if I go into the drive and run it manually), and FOC refuses to reinstall without EAW being reinstalled. Thanks! 97.125.84.72 (talk) 16:47, 18 August 2010 (UTC)

Sorry I don't know the answer to your question, however it's possible it's not just registry keys it needs, but also specific files in the Windows system directories. I'd simply contact LucasArts though, it might be a problem with something else that's stopping you from being able to install the game, but even if it is actually is a faulty disc they have a disc replacement policy where for only $5.00 USD per disc they'll swap your broken disc for a working one.  ZX81  ^talk 18:48, 18 August 2010 (UTC)

I solved this problem myself several days ago and am explaining here so that anyone else with the same problem will no what to do. I successfully installed Empire at War with another disk, then looked at the registry info. The registry key for the original EAW was "HKEY_CURRENT_MACHINE\SOFTWARE\LucasArts\Star Wars Empire At War\1.0\", containing: string "CD Key" with the CD key as its value; string "ExePath" as the path for the sweaw.exe executable file; string "Launcher" with the path for the launcher as its value; DWORD "Installed" with value 1; and DWord "Revision" with a value of 10105 (2779 in Hexadecimal). I created another registry key, "HKEY_CURRENT_MACHINE\SOFTWARE\LucasArts\Star Wars Empire At War Forces Of Corruption\1.0", and put in the strings for "CD Key" with my Forces Of Corruption CD key and "ExePath" with the path for swfoc.exe, and the DWord "Installed" as 1. It worked; although I used a noCD cracked executable and won't verify whether it would work without one because I fear SecuROM, which is deployed with the unaltered executables for FOC or EAW. 97.125.84.72 (talk) 06:31, 27 August 2010 (UTC)

Trying to install an MSDOS program on Vista

I just now downloaded the Shareware version of the original Duke Nukem game from http://www.3drealms.com/duke1, and upon opening the resulting zip file after completing the download, a window with a warning message appeared. Entitled "16 bit MS-DOS Subsystem", the window gave me the following text: "This system does not support fullscreen mode. Choose 'Close' to terminate the application." Any idea how to get this program to install on Windows Vista? Nyttend (talk) 19:35, 18 August 2010 (UTC)

DOSBOX 82.44.54.4 (talk) 19:42, 18 August 2010 (UTC)

Program is downloaded, and I've gotten it to work; thanks for the pointer. However, I'm now confused: how do I tell it to run the install program, or how do I tell the install program to run in Dosbox? I've looked and failed to find a "Run with" command when I rightclick on the install program in My Computer, and I can't remember how to work DOS; the readme for Dosbox doesn't seem to have a how-to-run-DOS element. Sorry if there's an obvious answer to my problem; I just can't think of how to do this. Nyttend (talk) 21:29, 18 August 2010 (UTC)

It will probably work if you just open the dosbox prompt and type the name of the executable, with its full path (e.g. c:\dowloads\duke.exe -- Finlay McWalter ☻ Talk 21:32, 18 August 2010 (UTC)

The program is called "INSTALL.EXE" and in a folder named "DUKE", but typing C:\DUKE\INSTALL.EXE results in a message of "Illegal command: C:\DUKE\INSTALL.EXE". Do I have to type something before the full path? "run" and then the path resulted in a message of "Illegal command: run". By the way, the readme says that I must follow a "mount" command; I don't understand what that does, but I've followed the readme's instructions and gotten the results that it said I should from that. Nyttend (talk) 21:43, 18 August 2010 (UTC)

You don't use RUN or anything, you just type in "INSTALL.EXE" after you have mounted the right directory as a drive in Dosbox. --Mr.98 (talk) 21:46, 18 August 2010 (UTC)

If Dosbox on Vista works the same as it does on OS X, what you do is install Dosbox, then you have to "mount" the directory with the program as a virtual drive within Dosbox (e.g. "MOUNT c d:\yourprograms\duke" makes it so that the C:\ drive in Dosbox corresponds to the folder on your D: drive as indicated). Then you run it from within Dosbox (e.g. "c:\duke.exe"). If you have forgotten your basic DOS commands, type in HELP and it'll give you them. --Mr.98 (talk) 21:45, 18 August 2010 (UTC)

Okay, it installed; the program isn't running properly, but I suspect that it's a compatibility issue. I'll try running it on an XP computer. Thanks, especially, for the HELP command; I had no idea that there was such a thing, but I was wishing that there were. Nyttend (talk) 23:10, 18 August 2010 (UTC)

There might be special Dosbox settings that will help. ("Dosbox -- all of the old frustrations of Dos, today!") I tried Googling "Duke Nukem Dosbox," and what do you know, someone has written a guide on getting it to work. Now some of this is about the CD-ROM version and probably doesn't apply, but I thought maybe it'd be a start. The Dosbox FAQ actually says specifically that it does run, but you have to be careful about selecting your graphics settings, because Dosbox is emulating the entire PC at once, and can't necessarily do it as well as the original hardware. This page has more specific .conf settings that might be of help. From the looks of things, Duke Nukem is a little hard to get started, because it — in its own day — pressed CPU resources pretty hard, and emulating that can be a little tricky. It seems do-able though. Good luck. --Mr.98 (talk) 01:14, 19 August 2010 (UTC)

Dammit, I should have posted this as soon as I saw your question and saved you some hassle. The easiest option is NOT to use the DOSBox directly but instead use one of the many graphical front-ends that have been developed for it. I'm partial to D-Fend Reloaded because of its nice interface listing all your games in the main screen one below the other, kinda like MAME does with arcade games. It has a "Add game" Wizard which takes you step by step through the process of adding a new game to the list (including the setup.exe or install.exe file), and its mouse-over tooltips are mostly quite helpful and explanatory. Zunaid 21:21, 19 August 2010 (UTC)

That's pretty good to know in general, thanks. --Mr.98 (talk) 16:31, 20 August 2010 (UTC)

Latest version of Netscape (and how to speed up netscape)

Hello there, I am using Netscape 9.0 Beta version 3. Is it the latest version? I am also trying to speed up the browser. So I found this (ehow.com/how_6001169_speed-up-netscape-navigator.html website). But the problem is, options mentioned in that article is not present in Netscape 9.0, for example, "Network Connections.", "Preferences" and "Connections" tab. Where could I get this option? thnaks--180.234.38.102 (talk) 20:52, 18 August 2010 (UTC)

No, the latest version of Netscape was 9.0.0.6 (from February 2008). If you are not already aware, Netscape is no longer actively developed. As explained on that history page, and our article Netscape Navigator, the technology that drove Netscape went through some complicated business dealings and ultimately emerged as the core for the Mozilla project. The newest version is Mozilla Firefox, Version 3.6.8. Nimur (talk) 21:10, 18 August 2010 (UTC)

(edit conflict)The most recent version is Netscape Navigator 9.0.0.6, released in February 2008. Beta 3 was released in August 2007, making it three years old now. If possible I'd recommend upgrading to a more modern browser. A more recent browser just might run faster. If not, those changes should still be possible with a new browser. Except possibly the first option, which I haven't seen in any browser that I've used. Reach Out to the Truth 21:13, 18 August 2010 (UTC)

It's probably best to update to the latest version of Netscape (9.0.0.6) or to switch to firefox. Netscape was abandoned in 2008, and is essentially replaced by firefox.Smallman12q (talk) 15:27, 19 August 2010 (UTC)

Wikipedia has a problem

Whenever I begin typing the URL to Wikipedia in Firefox, it will automatically suggest "en.wikipedia.org". However, the stored headline for that page is "Wikipedia has a problem". While it is definitely true that Wikipedia has its problems, this was not the headline of the page when I most recently visited it. It has been like this for quite a while, and I wonder if there is a way to fix this without purging all the stored URLs. Thanks, decltype (talk) 21:25, 18 August 2010 (UTC)

According to Wikipedia:Bypass_your_cache#Mozilla family hold shift and press the reload button to bypass your cache. Taemyr (talk) 21:33, 18 August 2010 (UTC)

Thanks, I've purged my cache but it didn't help. Perhaps my question was poorly worded — It is only in list that automatically drops down when I begin typing an URL that the headline is wrong. Regards, decltype (talk) 21:39, 18 August 2010 (UTC)

It sounds like a bookmark thing. Go to Bookmarks -> Organize bookmarks, search for the wikipedia link, highlight it, and at the bottom of the dialog box there should be some text boxes. Under "Name", change it to whatever you want it to say 82.44.54.4 (talk) 21:49, 18 August 2010 (UTC)

It's not a bookmark thing. Mine used to do that because of the recent serverdeath, but it's stopped doing it. sonia♫ 22:59, 18 August 2010 (UTC)

I'm not sure this will work, but clearing the entry may solve the problem. Start typing as you have been doing. When the mislabeled suggestion appears, use the down arrow to highlight it and then press the delete (DEL) key. Hopefully that will clear the entry and Firefox will get a new title the next time you visit the page. -- Tom N (tcncv) talk/contrib 00:23, 19 August 2010 (UTC)

Thanks all. Tcncv's suggestion kinda worked. The entry is gone, but it is not getting readded when I visit the URL in question. Not a big deal though :) Regards, decltype (talk) 04:36, 19 August 2010 (UTC)