Software-Generated Paper Accepted At IEEE Conference 235
schlangemann writes "Check out the paper Towards the Simulation of E-commerce by Herbert Schlangemann, which is available in the IEEEXplor database (full article available only to IEEE members). This generated paper has been accepted with review by the 2008 International Conference on Computer Science and Software Engineering (CSSE). According to the organizers, 'CSSE is one of the important conferences sponsored by IEEE Computer Society, which serves as a forum for scientists and engineers in the latest development of artificial intelligence, grid computing, computer graphics, database technology, and software engineering.' Even better, fake author Herbert Schlangemann has been selected as session chair (PDF) for that conference. (The name Schlangemann was chosen based on the short film Der Schlangemann by Andreas Hansson and Björn Renberg.)"
proving my point... (Score:4, Interesting)
...that peer reviewed journals (at least in computer science) are crap. 1) peer review is an old boys network, 2) people don't look at substance, they look for fancy buzzwords of the month and equations that look hard (you're rewarded for the more convoluted your paper is!), and 3) the way the system is setup, 99% of what is published is crap...people at universities and labs are forced to produce as many publications as possible to get promoted. It would be unfair of me to say that all of it is useless, but it's definitely inefficient. Look at where the great ideas in computer science and software development come today...they come from the community through things like open source (e.g. Linux, BitTorrent, etc). The academic community just rides on their coattails...
Sadly... (Score:4, Interesting)
This SCIgen system quite resembles how many undergrads I have seen write papers for many of their classes, not just computer science.
Nothing new (Score:5, Interesting)
https://secure.wikimedia.org/wikipedia/en/wiki/Scigen [wikimedia.org]
Does this program pass the Turing Test?
NSFW (Score:3, Interesting)
The links from the wikipedia article aren't quite safe for work. Anyone watching the short film Der Schlangemann at work should at least turn down the volume, or not watch at all. Lucky for me my boss would probably just laugh.
Re:Nothing new (Score:4, Interesting)
It makes me wonder... (Score:4, Interesting)
...does this mean that those who are supposed to review such things are either incompetent or don't bother with their job, or that many "professional science" papers are actually pure bullshit, so you can't tell the difference?
kind-of IEEE (Score:5, Interesting)
This is kind-of an IEEE conference. There are core IEEE conferences, which are run by the IEEE, which this isn't. Then there are other conferences (lots of them), which the IEEE sponsors in one way or another, and indexes the proceedings of. They often see the latter as a free (or at least cheap) way of getting their name associated with something that might take off. On the other hand, as this shows, it can get their name associated in the other sort of manner as well.
This seems to be a conference in China that was just founded, which leads me to believe the IEEE (like many stock investors) was duped in a rush to get their foot in the door of the Next Big Thing In China.
Lots of organizations do something vaguely like that, although the IEEE does seem to be worse than most. Even if you look only at their own, "branded" journals (IEEE Transactions on Foo), they seem to be founding new ones ever other week, which range in quality all the way from well respected in their field, to kooky. If they aren't careful, they're going to start getting an Elsevier-level reputation.
Re:proving my point... (Score:5, Interesting)
Just for an example of poor quality (though opposite to your comments about convolution), I was once handed a paper (I don't recall the journal or author, unfortunately, as this was several years ago) on fault detection in distributed networks.
The entire point of the paper was "If you send a request and you don't get a response back for a while, something probably went wrong." I read over it a couple times, hoping I was missing something that actually had substance to it. No luck.
That said, I don't think the academic community is entirely full of crap, or just riding on coattails. I do think a lot of that goes on, though, and I think it really pollutes the overall signal/noise ratio in the related journals- and from a distance, it does tend to just blur together into "crap".
Not a robot conspiracy (Score:5, Interesting)
Actually, if you look at the details, this "paper" was accepted into the poster session for the conference. I've been on enough technical program committees to know that the standards for poster acceptance vary quite wildly.
At some conferences, acceptance is done first and then papers are sorted into posters and presentations purely on the basis of what mode is most suitable to the material.
In others conferences, all the rejected papers are automatically accepted as posters. Why? Because conferences have expenses and to recover expenses they need attendees. Many institutions only pay for travel and registration if their employees have papers accepted at the event. So, to allow people to attend, they have to accept more papers than they might want to. With the rise of for-profit conference organizing companies, there is even a profit motive in some cases.
There is a vigorous debate within the IEEE whether such "pity accept" papers should be allowed into IEEE Xplore -- the long term archive of papers maintained for posterity. The decision is left to the conference organizers with the idea that including obvious junk in the archive actually has relatively low social cost since nobody would ever cite it or rely on it. So who cares. Others are embarrassed to have such crap in the company of more important work.
Re:Why is this shocking? (Score:3, Interesting)
Since the number of problem-solvers is independent of the number of problems, and problem-solvers can examine multiple solutions to the same problem, along with a limited range of solutions for many problems, you can expect the number of publications to exceed the number of problems and the number of problem-solvers. However, you are correct that merely increasing the quantity of papers (which is all the current rules do) will cause the quality to suffer. The total thought put in to N papers over a period of time t cannot exceed the total amount of thought the brain can output over time t.
Sure, natural multitaskers will be able to better exploit the total amount of thought the brain can exploit when N exceeds 1, but if N exceeds their threshold, the quality suffers. For the rest of humanity, where single-tasking is the rule, N absolutely has to be 1.
Current funding rules for academia and research labs mean that quantity is profitable, quality is not. That is exactly the wrong way to get any real work done and is partly why papers on hyperdilution and test-tube cold fusion are serious money-spinners. They take no effort to write, get cited lots (even if by debunkers - doesn't matter, since funding is a function of the absolute number of citations and not by whether the citing papers agree), and grab the attention of potential external sponsors who couldn't tell a good paper from a confetti'd dingo's kidneys.
Re:proving my point... (Score:5, Interesting)
There is a lot of crap, I agree, because many profs save time by having students write papers, and when you talk about riding coattails, the profs just put their name along with the student's name on the sometimes quite long list of authors. On the surface, there is a "structured" way of writing a paper so that it passes muster. After that, deep thoughts are typically not described in detail. The reader is "supposed to" have enough IQ or education to follow along.
Is it an old boys network? Perhaps, but I don't really think so. The system works, as in it is maintainable rather than functional, rather than perfect, because there is enough money to keep below average students. After all, if you have only above average students, you will end up with below average students, below a larger average, but then you need above average profs and above average budgets. Just as subprime was aided by mandates to provide poor people with access to housing, universities have to admit bad performance.
The problem is a lot of writers tend to gravitate to lower standards, partly to save time on writing, and partly because the system makes papers suck so badly that it becomes easy to become published - it's an artificial way of paving the road to academic recognition. Reviewers are inundated with garbage. They can't reject as many as they want because they have a "quota". Also, some people need an incentive to become researchers, trying to achieve something risky, and they aren't going to stay in the field if their papers keep getting rejected.
There has to be a happy ending - if you want to figure out something, don't just search the literature. The exact answer isn't there. You have to solve the problems yourself and skim the papers for little insights into techniques or results.
Re:Not a robot conspiracy (Score:5, Interesting)
Actually, if you look at the details, this "paper" was accepted into the poster session for the conference.
Most poster sessions consist of abstracts that are *rejected* for a speaking slot. They usually don't need to pass any more muster than to be properly formatted. A stylesheet takes care of that--no AI needed. The final say after rejection for a speaking slot basically comes down to an administrative assistant's being able to know which side is up and to have enough room in the left margin for binding. The idea is that a poster session attendee will have a liver presenter in front of the poster to explain the poorly written abstract.
Re:It makes me wonder... (Score:5, Interesting)
I took a grad level course at the end of my degree in which the prof tried to subtly communicate this to the grad students.
Each week we had to read and write a short review on a paper that had been peer reviewed and accepted into some CS or Eng journal somewhere. The topics were pretty broad, but all had something to do with the internet, or communication (the focus of the course).
At the end of the course, the professor revealed that he had purposely selected papers such that one third were considered today to be 'good' papers, one third were considered to be valid, but poorly written, and one third were considered to be pure bunk but well written.
He then posted a graph showing how people commented on each of the papers.
Not surprisingly, nearly every student reviewed all of the papers in a positive light.
I'm pretty sure that the professor was trying to teach a bit humility to the grad students. He also succeeded in proving "...that those who are supposed to review [the papers] are either incompetent or don't bother with their job, or that many "professional science" papers are actually pure bullshit, so you can't tell the difference?"
Review system flawed (Score:5, Interesting)
Re:Not a robot conspiracy (Score:3, Interesting)
Re:proving my point... (Score:3, Interesting)
I'm not a computer scientist, so YMMV, but I frequently act as a peer reviewer for several agricultural journals. We have no quota with respect to acceptance or rejection, and are not compensated monetarily or otherwise for our service as reviewers. I've rejected papers before, and it seems like I've rejected more as time goes on. I've even had a couple of my own papers rejected -- not every paper is a winner. One thing I have noticed is that I review for a journal with a very high impact rating, and there has been a noticeable increase in mediocre-quality papers as people chase the impact rating.
Re:Sokal affair Redux? (Score:2, Interesting)
At least Social Text wasn't a peer-reviewed publication [tohoku.ac.jp]. IEEE doesn't seem to have that excuse.
Re:Not a robot conspiracy (Score:3, Interesting)
It shouldn't have just been denied an oral presentation, it should have been caught by the program committee and never reviewed. You can't read 3 sentences of that abstract without knowing that it's garbage.
Presumably someone DID review this and deny it an oral, but didn't follow up with the program committee to make sure it was pulled entirely.
I've never been to a conference which pity accepts papers. CVPR, a IEEE conference on computer vision, has a 25% acceptance rate for posters. I think this paper is quite an embarrassment to IEEE.
Meh. As others have noted, it was for a poster session. This conference isn't in my field, but at the conferences I've been to in my field (astronomy and astrophysics), pretty much anything gets accepted for poster sessions. At AAS [aas.org] meetings, I've seen particularly wacky posters in extragalactic astronomy and cosmology all clumped together in a kind of ghetto; and back when I was a grad student, during free time between oral sessions or at the end of the day, someone among my friends and I would say "hey, let's go look at the crazy stuff" and we'd take a look at the posters about space potatoes [cs.hut.fi] or the Plutonium Atom Totality [wikipedia.org] or whatever. On occasion, I've even seen oral sessions -- typically one of the last ones on the last day -- devoted to something like "Speculative Ideas in Cosmology" with some of the nuttier talks tucked in there (as well as ones which are almost certainly wrong, but aren't in the same league of crazy as we're talking about, unfortunately).
hardly the only thing that makes it different (Score:3, Interesting)
Another thing that makes it different is that this is a no-name startup conference in China with editors and a program committee nobody has ever heard of, whereas Sokal got his paper accepted to a well-known journal in the field, edited by some of its luminaries. If this paper had gotten accepted to say, Communications of the ACM, and fooled Donald Knuth into thinking it was genuine, it would've been more analogous.
Re:I For One... (Score:2, Interesting)
Re:proving my point... (Score:3, Interesting)
I don't buy it. Academia may have some issues, but there are certainly things that Academia is good for and some things that an open-source-type community is good for.
It has been a long time dince academicians have developed a new pulic worthy OS, but I would be highly surprised to see the public at large develope something as difficult, complex, abstract and important as the PCP Theorem [wikipedia.org] (probably the greatest recent comment on the P != NP conjecture--for those of you who are interested, the theorem says that supposing P!=NP, then there is a limit as to how close an approximation algorithm can approximate a solution, conversely, if an approximation algorithm can come closer than that limit, then P=NP).
Re:I For One... (Score:1, Interesting)
The best phrase from this paper is:
"Continuing with this rationale, we removed more 2GHz Intel 386s from the KGB's game-theoretic
cluster to understand our desktop machines."
It's good to know that, in the study of e-commerce simulation, one can always depend on 2 GHz 386s from the KGB game-theory cluster. This was clearly needed to understand a desktop.
I'll find out soon enough if the IEEE watermarks the PDFs with the username and IP of the original downloader - since here's a download [rapidshare.com] link.
Merry Christmas, Happy Holidays,
Satan Claws.
Re:I For One... (Score:3, Interesting)