My notes on Intel's Transactional Memory (it's like a cmpxchg64b)
Fri, 10 Feb 2012 01:08:00 +0000

People are excited that Intel is adding "transactional memory" aka "TSX" features in 2013. Ars Technica has an article here, but it feels like gobbledeegook to me. I'm not sure I can provide a less confusing description, but as a programmer who writes multicore code core, I think I can provide a different description.


It's not about making synchronization "easier" (as Ars Technica claims), but about making synchronization more "scalable". The problem with multicore code is that, after a point, adding more cores doesn't make the software any faster. That's because, after a point, the synchronization overhead needed by adding additional cores exceeds the additional processing power. This is known as "Amdahl's Law". A lot of multithreaded software can use 2, 3, or maybe 4 cores, but must software fails to scale to the 12 cores you get on a high-end Intel desktop.

Transactional memory reduces the cost of synchronization, and thus, makes software more scalable.

Average programmers won't use transactional memory themselves, but instead, it's the operating system, Java VMs, and assorted libraries that will use it. The programmers will just notice that their code now scales better.

An example of this is the code "atomic {x++; y++;}" on the Ars Technica article. That's precisely the sort of thing you can't do with TSX instructions. TSX transactions work on a 64 byte cache line. Thus, the above code snippet would only work if 'x' and 'y' were on the same cache line. These complicated details means it's unlikely to that high-level language constructs will matter. Instead, programmers will see low-level constructs, such as the current "intrinsics" used for SSE/AVX code that map from C/C++ to x86 assembly instructions. Programmers who write operating systems or libraries will use these instrinsics, but most programmers won't.


At a lower level, what these instructions really do is change how programmers use "spinlocks" and "wait-free algorithms".

A "spinlock" is when an additional number is used to synchronize access to memory. Before messing around with a chunk of data, code first changes the spinlock from '0' to '1', then messes around, then changes it back from '1' to '0'. If another thread comes along during this time, it notices that the spinlock is '1'. The other thread then "spins", repeatedly checking the spinlock value over and over again until it goes back to zero.

Notice that in some cases, the above logic wouldn't work. If two threads hit the start of this code at precisely the same time, both will see that the spinlock variable is '0', both will change it to '1' at the same time, and both will proceed, causing bad things to happen. Thus, this change is done in what's called an "atomic" operation. On the x86, this is done with a "lock cmpxchg" instruction. This locks out other processor cores from changing that value at the same time. In this scenario, one core will succeed in changing the value from '0' to '1', but the other core will fail, and know to start spinning.

The "wait-free" techniques take a different approach. Instead of using a single atomic instruction to synchronize access, they use a series of atomic instructions. This allows both threads to make forward progress, without forcing one thread to stop and wait for another.

Whether "spinlocks" or "wait-free" is better depends on a lot of factors. Wait-free is better from a theoretical point of view, but since atomic instructions are slow (30 clock cycles, or L3 cache speed), wait-free can be slower in practice.


So here's what TSX does. If you are thinking in terms of "spinlocks", it makes the code more efficient. Change your data so that it fits within a single cache-line, then write your "spinlock" as normal, but with a TSX extension ("xacquire lock cmpxchg", instead of "lock cmpxchg"). What happens is that, on older processors, the "xacquire" is ignored as a NOP (its value is 0xF2, which is meaningless to older processors in this context). On newer processors (2013 or later), the processor keeps track of what's really happening, and makes it more efficient. If two threads protected by that spinlock don't actually write into each other's data, then it's as if the spinlock didn't exist.

Better yet, if you are thinking in terms of "wait-free" code, you can do even more useful things. Wait-free code is written using something called "CAS2", or "double compare and swap". This compares two values simultaneously, and if both values match, swaps both to new values. This is often done swapping two pointers (to two new pointers), or a pointer plus integer. On 32-bit x86 this is the 'lock cmpxchg8b' instruction for swapping 8 bytes; on 64-bit x86 this is the 'lock cmpxchg16b' instruction, for swapping 16 bytes.

Because TSX operates on a cache line, which is 64 bytes wide, this effectively becomes a "lock cmpxchg64b" instruction, or CAS8. This makes wait-free code dramatically more efficient.

Conclusion

So the upshot is this. The problem is that software can only take advantage of a small number of processor cores before synchronization becomes too expensive. The new "transaction" features reduce the cost of synchronization, and hence, allows software to take advantage of more processing cores. There are two high-speed synchronization: spinlocks using a single atomic and wait, and wait-free that uses multiple atomics. In particular, it extends the old instruction that allowed 16 byte transactions (cmpxchg16b) to something that allows 64 byte transactions (think of it as "cmpxchg64b", but more efficient).

---

From the author of the Ars Technica piece:



No, I have read the Intel docs (the spec on two Intel blogposts). The difference is that that the Ars Technic posts just repeats them, going bottom-up, starting with the details. I'm trying to go top-down: starting with how people code today, and how the transactional features change that. For example, if you've every used the existing transactional feature "cmpxchg16b", you realize how much nicer life will be with these changes.

Some IDS comments
Mon, 06 Feb 2012 21:52:00 +0000

I saw this go across my twitter feed:




This appears to be a simple feature, but it's actually quite complex to do right. I'll explain it here. I implemented this feature back in 1998 in the "BlackICE" IPS (now sold as IBM Proventia), and it works really well.

The HTTP response code is something like "200 Ok" or "404 Not Found". If the attack was sent to a server, looking at the response from the server can indicate whether the attack succeeded or failed.

You'll notice a problem here: the response code comes after the intrusion. Should the IDS report a second event for the response code, or should it hold onto the intrusion event for a couple seconds waiting to see if there is a matching response code it can report with the intrusion?

The BlackICE solution was to do both. Each event is marked with a unique ID. The intrusion on the request side is reported first. Then, when the response code is seen, a second intrusion event is reported with the same unique ID as the initial event.

What happens is that the system "coalesces" the event. Events are held for a time on the sensor in an outgoing queue, arrive on the console and are stored in an incoming queue, and are then stored in a database. The update event chases the initial event. If it catches up in the outgoing queue, the event is coalesced on the sensor. If it catches up on the console's incoming queue, it's coalesced there. Otherwise, it's coalesced in the database. The user-interface (which is a front-end on top the database) will also coalesce events.

My point here is to point out that this simple change, depending on how you implement it, can affect the entire ecosystem. It was a major change for ISS when they bought my company and replaced their RealSecure with BlackICE and renamed it Proventia. BlackICE was highly modular, so swapping in new brains into the sensor was a trivial task. Redefining what it meant to handle events, though, took longer to integrate.

Another problem is how you report the data. BlackICE was a protocol-analysis system rather than a pattern-matching system. What that means is that, like Wireshark, it would pull out the contents of fields and report them along with the event. The number of protocol fields decoded by Snort is fixed at a small number (less than 20). The number of protocol fields reported in BlackICE events number in the thousands, and new ones appear every month. Snort defines it's fields statically, whereas BlackICE has space for arbitrary pairs. I've searched for an event format in Unified2 that supports arbitrary pairs, but I can't find one. Nor do any consoles have the feature of reporting on arbitrary fields, or searching for events based on the contents of such fields.

Again, it appears that adding HTTP response codes requires a change in the entire Snort ecosystem.

Finally, there is a difficulty about what, precisely, the response codes mean. For some events, a "200 ok" means the attack succeeded. For others, "200 ok" means the attack failed. In still others, "200 ok" is inconclusive: ColdFusion always returns "200 ok" regardless of what happens. For other events, any HTTP response code means the attack failed, and a RST indicates the attack probably succeeded.

When adding HTTP signatures to BlackICE, only 20% changed their priority based on the HTTP response code. For the rest, we simply reported the HTTP response code as part of the event, and let the analyst on the console deal with it. For example, if the analyst saw a large number of "directory traversal" events, the analyst could sort by response code to see which ones succeeded and which ones failed.

Conclusion

You can actually do the basic feature today with Snort. The latest version allows rules to be triggered on HTTP response codes that can be dynamically enabled after another event fires. It's not pretty, but for specific HTTP events where you absolutely must know the response code, the technique will work. It's one of the extraordinary things about Snort: it's flexibility means that somebody might figure out how to do something that somebody else else thinks is impossible.

But to do it right (as I define "right"), you need to redesign the entire Snort ecosystem. Event coalescing and decoration, they way BlackICE/Proventia does it, is extraordinarily useful in so many ways, so I recommend that Snort make the change.

---

What would the change look like?

A simple solution would be to define a new type for the "extra data" field, where the content of that field will consist of 8-bit "name length" followed by a name, followed by "value length", followed by the value data, followed by yet more name/value pairs. This would fit neatly into the Snort output without disturbing the downstream ecosystem.

The next step would be to start decorating events. For example, for those protocols that have a "username", like SMB or FTP or HTTP or MSRPC, parse that username, save it per TCP connection, and when events are reported, include the "username" in this extra data.

Now that you have arbitrary name=value pairs in events, think of ways to use them. For example, consider the following:

alert tcp any any -> any any (msg:"Extract Foo"; content: "foo="; pcre:"foo=(?P=[A-Z]*)";)

What this arcane magic means is search for the pattern "foo=" in a TCP stream, then extract the following upper-case text, then report that text with the name/value pair BAR.

Ok, now that you're producing events with name/value pairs, the rest of the ecosystem has to catch up. This largely means a massive change to the database schemas to add a table with the columns "EVENTID=int, NAME=varchar, VALUE=blob".

In the user interface, you now need to allow people to "inspect events" to view any name/value pairs attached to it. You also need to be able to display table of events, showing the different values of the name/value pairs. You also need to be able to search for events based upon the name/value pairs, such as:

SELECT * FROM events WHERE event_id IN (SELECT event_id FROM namevalue WHERE name='BAR');

Why we have jobs in cybersec
Thu, 02 Feb 2012 22:37:00 +0000

I just got an email from my accountant:

Attached, please find your 2011 Tax Organizer, which has been password protected. The Password is the FIRST FOUR digits of the taxpayer's social security number.

This seems reasonable. After all, your card for ATM machines has only a 4 digit PIN number. In addition, since the LAST 4 digits is so often used, many people know it, so they chose 4 digits that somebody else wouldn't know.

But of course, the problems with this are obvious to any professional.

There are three reasons why 4 digits work for ATM machines, and why they don't work here.

• The ATM card itself the PRIMARY security, the PIN number is only SECONDARY.
  
• Guessing the PIN number is "online" (you can only guess a few numbers before the ATM machine eats your card), but PDF guessing is "offline" (you can make as many failed guesses as you want).
  
• The third reason things are different is that stealing money from an ATM is limited to only a few hundred dollars, whereas documents from your accountant can lead to loss of all your money.
  

I can pay my neighbor's kid $20 to sit in front of a computer for a couple hours trying all 10,000 combinations until they guess the right password. The kid might get smart and google social security number prefixes and reduce the number of attempts by quite a lot. Indeed, if he could figure out where I was born, he might reduce his search to only a few hundred attempts, because the first three digits are assigned by which state you are born in. Which is why people ask you for your last 4 digits rather the first 4 digits, because they are so easily guessed.

Or, I can download free software to do it for me. I downloaded this program and after 2 seconds of crunching numbers, it came up with the right password:


(This image is edited, of course, my SSN# does not actually start with "5967".)

So, what's the right solution? You can't send an encrypted PDF and the password in the same e-mail (as some people do), because then hackers yet again and decrypt the PDF. Instead, you have to exchange passwords "out-of-band", such as on the phone or when you visit the office. The encryption is only as strong as the password, so you have to choose a long one (more than 12 characters that are hard to guess).

The REAL correct solution is for vendors to better integration PGP or S/MIME into email systems. PDF encryption was chosen in this case because it's built-in. Likewise, generating public/private keys should be built into every e-mail system -- but it's not.

January 18: SOPA blackout day
Wed, 11 Jan 2012 23:42:00 +0000

Reddit has decided to blackout their site on January 18 in protest against SOPA and PROTECT IP. This blog will, too.

This blog is hosted on blogspot.com, so I can't pull the plug on it. What I can do instead is simply change the template so that the the background is black and the foreground is also black. I've done this for the demonstration site http://sopa-protest.blogspot.com. You can see that all the articles, such as this one and this one, have the same template, and thus have the same blackout effect, so I don't need to edit the articles individually to cause the blackout. After January 18th, I'll simply change the template back again.

Thus, the steps are:

1. SAVE the original template first!!!
   
2. Change the template ("Edit HTML") so that the text is black-on-black, so nobody can read it.
   
3. Add the protest message to the template, such after the tag.
   
4. Save the new template at 8am on January 18, 2012.
   
5. Restore your old, saved template at 8pm January 18, 2012.
   
6. This November, send donations to the competitors of those politician who voted for SOPA.
   

(Of course, only a couple thousand people will notice the difference on our blog, most of whom oppose SOPA anyway, but the symbolic gesture is still important).

---

Update: Here is a list of other websites blacking out on Jan 18.

---

Update: You can easily change your Twitter picture to include a SOPA reference here: http://www.blackoutsopa.org/

---

Update: Here is a great link http://www.techdirt.com/articles/20120111/09293817377/as-sopapipa-becomes-toxic-frantic-congress-test-runs-dropping-dns-blocking-provisions.shtml that discusses how the problems with SOPA aren't simply with the well-known "DNS blocking" issue. It's a wide range of issues that gives too much power to copyright holders, and forces websites to block content because of liability risk.

---

Update: BTW, it's interesting how Reddit has become the new center of geekdom. Even though this blog has been Slashdotted a few times, far more traffic has come from Reddit. I wonder if Slashdot plans to similarly blackout their site in protest.

This message will self-destruct in five seconds.
Wed, 11 Jan 2012 03:27:00 +0000

(Warning: Spoiler Alert ahead... or maybe not. I mean, it's not like there's some big Shyamalanesque plot-twist at the end of these things...)

The other day a friend said, "You have to go see Mission: Impossible - Ghost Protocol! You will love it, ya know, because you're in security." I'm not really the type that goes and sees every action movie, but I was sufficiently intrigued by the promise that the fourth installment of the series might be a hacker flick. Those are always good for a sobering insight into what Hollywood thinks of our industry or for a laugh. So I went... and I loved it! It was the gadget filled awesomely insane tapestry of extreme action and suspense that we all have come to love and expect from Tom Cruise.

But it didn't seem to me to be a hacker flick. So I messaged my friend and I said, "The gadgets were by far the best in this movie. The story was the most appealing. The actors all had great chemistry. But that isn't why you said I would like it, so please explain, why did you call that a movie about infosec?" He then began recounting all of the scenes where Old Man Cruise has to rappelle from something or dive off something and get something out of some ridiculously locked room. But what he of course noticed that I had been too dazzled to see was that the real heavy lifting in those scenes was done by the team's standard issue hacker character (Simon Pegg). Tom has to go into the vault to get the microfiche (really, still??) but Simon is the one that gets that door open.

The most interesting part though is how the hacking is done. In a cruel twist of fate and conspiracy from the highest levels, the president initiates "Ghost Protocol" and the team becomes exiled with no access to the Carnivore-like CIA network that usually makes things like breaking the encryptions Hollywood-quick. So they're forced to kick it old-school and do a pretty nice variety of physical penetration hacks.

MI:4 has reminded me how effective the physical security attack really is. While today's military grade firewall may be Fort Knox at keeping people out of the tubes, there's really nothing that's going to stop a hacker if they're sitting right in front of the machine. Or if their increasingly disgruntled team leader is sitting in front of the machine with a pocket router after having scaled the sheer side of the tallest building in Dubai using only a suction cup and a fire hose. Or if the guy on the team who was never part of the plan that has to slide down an HVAC shaft into a subterranean server room that without the cooling system has become "an oven", and by the way the walkie-talkies aren't working and the bad guys just cut the satellite feed, is sitting right in front of the machine. Or if the plucky new female agent with a grudge and something to prove floats a balloon holding a wireless connection device over a wall to get into the signal area.... Well, I guess they can't all be extreme, but it shows the excellent point that if your physical security strategy doesn't cover the 50 feet underground and the 15,000 feet of air space above it, you're doomed. (Don't worry the plucky female agent gets extreme redemption when she completes one of our other favorite old-school physical hacks, the 'beating someone with a $5 hammer [xkcd] until they tell you the password' technique.)

Oh, and also everyone on the property should probably be assigned a dog because people are incredibly dumb.

Multithreaded teaches the wrong lessons about multicore
Mon, 09 Jan 2012 22:22:00 +0000

This blog-post compares two open-source "packet logging" programs. These are simple programs that log network traffic directly to the disk. That blog-post finds that the multithreaded program is a lot faster than the single-threaded program, confirming people’s prejudices that in the modern world with multicore systems, multithreaded is better.

But the results are suspect. It finds that TWO-threaded program is SIX times faster. That doesn’t make sense. If the issue were truly just "multithreaded vs single-threaded", then at most we’d expect at most a two-fold increase, not a six-fold increase.

Instead, the real problem here is the way that the application has to "wait" on either the network or the disk. One way to solve this waiting is to put the network portion on one thread, and the disk portion on another thread. That’s what Gulp does. It’s many times faster than Daemonlogger even on computers with only a single processing core.

But multithreaded is only one way to solve this problem. Another way would be to use asynchronous APIs, and/or larger buffers. It’s the same way that single-threaded programs have long dealt with "waiting". like "C10K" web-servers that might be only single-threaded.

The true reason Gulp is faster has nothing to do with its multithreaded nature, but the way it cleverly uses Linux APIs in order to get out of the way. Network adapters want to DMA packets directly into a buffer at full speed, bypassing the CPU and operating system kernel. Disk adapters want to DMA data directly from memory at full speed, likewise bypassing the CPU and kernel. Today’s hardware can easily do this at many times 10-gbps speeds. The problem is that today’s operating system kernels get in the way. The trick to making this work is to figure out just the right operating system APIs to trick the kernel into getting out of the way. The reason Gulp is faster is because it does a better job getting out of the way than Daemonlogger, not because it’s multithreaded.

More the point, Gulp still fails at being "multicore". Computers have been stuck at 3-GHz for the past decade, instead of getting faster, we now get multiple cores. Gulp scales to 2 cores, but not 12 cores. It’s no faster in a 12 core system than a 2 core system. (My laptop has 4 cores, my desktop has 12 cores).

The problem we face today is that people think "multithreaded" means "multicore". It doesn’t. Multithreaded means running DIFFERENT tasks on a SINGLE core, like how Gulp runs one thread for capture and one for logging to disk, making it faster than Daemonlogger even on a single core. In contrast, multicore programming means running the SAME tasks on MANY cores, so making something faster simply means adding cores. Gulp fails at this.

Most software that people hail as being "multithreaded" fails at being truly "multicore". A good example of this is the Linux kernel itself, which claims to scale to 64 core. It does, but only for selected applications and bencharmarks. Linux fails at being truly multicore for other tasks, such as packet-sniffing. A great many multithreaded applications fail to scale well on Linux.

Another example is PF_RING. It uses custom drivers to bypass the inefficiencies of the Linux kernel for 10gbps speeds, but then it uses "spinlocks" instead of "atomics" for synchronization, so it fails at being multicore. After about 4 cores, adding additional cores makes PF_RING go slower, not faster.

If you want a truly scalable system, instead of going "multithreaded", you need to cheat. Today’s packet-sniffing adapters (PF_RING, Napatech) can split incoming traffic into separate streams by hashing TCP/IP socket info. So exploit that. Buy a cheap 8-core system, use one of these adapters to create 8 streams, and buy 8 high-speed disks (like SSDs). Simply run 8 separate instances of Gulp/Daemonlogger, each bound to a core, stream, and disk. When you want to analyze the traffic logged to the disks, you’ll have to recombine the streams back into a single one, but that’s not too difficult, especially when you are using a system costing $2000 that would otherwise cost you $50,000.

Conclusion

That original blog-post confirms your prejudices that multithreaded software is inherently better than single-threaded software, an important lesson for today’s multicore computers. But, when you look deeper at it, you find that the results are suspect and that it teaches entirely the wrong lessons about multithreaded software. Gulp fails at being multicore every much as Daemonlogger does.

---

Historical note: BlackICE, the first IPS written in 1998, was a two-threaded system, with one thread for packet-capturing (using a custom driver that looks a like like modern PF_RING) and another thread for analysis. It had the same "producer-consumer" relationship that Gulp has. While it was multithreaded, it wasn't truly multicore, and did not scale past two cores. I don't know for sure, but I'm told that IBM (which now sells BlackICE as "Proventia") has converted the software so that it's truly multicore.

---

Update: One tweeter took exception to my terminology, since nobody else makes the distinction/comparision between "multithreaded" and "multicore" the way I do. But that's entirely the point. Multithreaded programming techniques were developed for decades for either single core systems, or systems with a small number of cores. Those techniques your textbooks teach you fail when you get to 12 cores, like I have on my desktop. Just because a program (like Gulp) is multithreaded doesn't mean it's solved the problem of running on all 12 cores (which it doesn't). Thus, just because something is "multithreaded" doesn't mean that it's truly "multicore".

I forget who, but somebody (Azul Systems) has created a hashtable using atomic operations that scales to 1000 cores doing insertions and lookups simultaneously. Now THAT is true multicore.

---

Update: The 'cheat' solution I mention above is how people run high-speed Snort, a painfully single-threaded IDS. I think it's a bastard solution to the problem, but it turns out, customers are actually quite happy with it. (Which I guess is another lesson: what matters is how much customers like the pork sausage, not how it's made).

So lets you test this by using something like 'tcpreplay' at 10gbps. You'll find that the solution doesn't appear to work. That's because using tcpreplay, you take packets captured from slow networks and replay them at much higher speeds. On slow networks, like your home 10-mbps connection, a single TCP connection can use up the entire bandwidth. When you replay at 10-gbps, a single TCP connection captured at 10mbps is being replayed at 10gbps, which causes it to be sent to single virtualized adapter, which can't handle more than 1.25-gbps.

Thus, when testing your cheated Snort solution, you now have two separate metrics: maximum network speed, and maximum TCP connection speed.

But a truly multithreaded/multicore solution might not doing any better. Packets on a TCP connection must still be processed in-order, so you can't have one core process one packet on the TCP connection while another core processes another packet. Instead, to truly speed up the single-TCP-connection problem, you'll have to have multiple cores working together on a single packet. That's a hard problem, because chances are good that synchronization overhead (even using lightweight atomics) will cost more than you gain. Thus, a cheating solution may actually perform better on this metric than the proper solution.

Either way, I hope IDS/IPS evaluators like NSS start measuring single-TCP-connection speed along with max-network-speed.

---

Update: So how can you fix PF_RING to be multicore? Well, a good lesson is how PACKET_RX_RING does it. Both similarly-named solutions do roughly the same thing: create a memory mapped user-space ring-buffer for incoming packets. PF_RING does this with zero copy at 15-million packets/second bypassing the kernel, PACKET_RX_RING does this with making kernel copies at 1.5-million packets/second.

Ring-buffers are easily synchronized in a producer/consumer fashion. If there is only one consumer, then no special synchronization is needed. If there is one producer and many consumers, then the consumers need to synchronize among themselves, but not with the producer.

PACKET_RX_RING, while slow because of interference with the kernel, allows wait-free synchronization. The 12 threads trying read packets simply do a __atomic_compare_and_exchange() on the "current packet" pointer (which in x86 will be a lock compxchg instruction). If the operation succeeds, the current thread owns the packet, if it fails, the current thread tries again OR goes to sleep. (This synchronization also implies thread scheduling, so that threads can go to sleep, causing CPU cores to go to sleep, consuming less electricity).

PF_RING, while otherwise fast, does numerous "spinlock()s". When trying to read a packet, threads will furiously spin consuming vast amount of resources, causing the system to slow down as you add more threads.

On Linux, the 'spinlock()' wait primitive is thought to be very fast, because it has the best "best-case" performance. If there is no conflict, it is just as fast as an atomic primitive. However, when there is a lot of conflict, because you have a lot of threads, it has one of the worst "worst-case" performances, because they will be furiously spinning using up system resources.

So the upshot is that PF_RING needs to get rid of all "spinlocks" and use "atomics" instead, so that 12 cores are faster than 11 cores, and so that it allows the application to schedule threads to go to sleep instead of furiously spinning. As with PACKET_RX_RING, you shouldn't need more than one atomic compare-and-swap per packet read from the interface.

(Note: These comments are from playing with PF_RING last year. I used one 10gig transmit adapter and another receive adapter. I used the built-in sample apps off of 'dna0' that allow you to specify the number of threads. The more threads, the slower the packet receive, 1 thread did about 12-million packets/second, 12 threads did 1-million packets/second. Looking in the open-source part of the code, I saw evil spinlocks. I didn't disassemble the closed-source part of the code in order to see why the spinlocks were necessary).

---

Update: Ten years ago, the x86 lock prefix forces an uncached memory transaction, which took about 250 clock cycles on a 3-GHz Pentium 4. Today, with integrated memory controllers, it causes a L3 cache operation, which can be as low as 25 clock cycles on a 3-GHz Sandy Bridge processor.

The upshot is that "atomic" operations were expensive in the era of "multithreaded" code, but have become ten times cheaper in the era of "multicore" code.

Internet is indeed a human right
Thu, 05 Jan 2012 20:23:00 +0000

Vint Cerf (former Founding Father of the Internet, and current Google lobbyist) says that the Internet access is not a human right. He is profoundly wrong.

The gist of his argument is that the Internet is just technology. It’s how we use this technology (for things like speech) that is the human right, not the technology itself. That’s the wrong way to look at it. New technology adds new complications that require clarification.

That's what happened with the printing press. Our founding fathers chose to enshrine technology in our Bill of Rights, by saying that "Congress shall pass no law abridging the freedom of the printing press". The invention of the printing press revealed new rights, new concerns nobody cared about until the printing press appeared. It's difficult trying to list these new rights without reference to the technology that enabled them. Instead of "right to publish", it's just easier to simply say "right to printing-press".

Vint Cerf says "It is a mistake to place any particular technology in this exalted category [human rights], since over time we will end up valuing the wrong things". The printing press disproves this -- even though actual printing presses are certainly becoming obsolete, the values they revealed are not.

You might be tempted to apply Cerf’s argument’s the printing press, and say that "freedom of speech" already covers "freedom of the printing press", but you’d be wrong. As history has shown, it’s not always clear how to map one right onto the other. Reasons why governments restrict speech are different from the reasons why governments restrict presses. The type of restriction against speakers at crowded protests are very different than the restrictions against printed agitprop pamphlets. Governments can restrict the printing press without, technically, infringing speech.

For example, government originally licensed printing presses. The reason was that the press introduced new economics. It cost a lot to setup the press for the first copy, but subsequent copies were very cheap. You could only pay back the original investment if you could sell a lot of copies. If two printers decided to print the same thing at the same time, then neither could recoup their initial investment, and both would go bankrupt. Therefore, some coordination by the government was "needed". This was the situation before 1709 in England. The abuse of that system, such as government censorship, forced the laws to change ("Statute of Anne").

You might point out that the First Ammendment actually said only "press" and not "printing press" (correct) and argue that it therefore only referred to newspapers (wrong). By "press" it meant all actions of the printing press, including printing things like the Thomas Paine’s Common Sense or the Declaration of Independence. The First Ammendment very much refers to the situation 100 years earlier when the English government controlled the printing presses.

Update: Eugene Volokh has a great discussion of this here (summarized here). Samuel Johnson's 1755 dictionary makes no mention of the newspaper industry when defining "press". It wasn't until after the First Ammendment was written that "press" started to be used for "newspaper industry".

We have the same situation today, where today’s copyright laws are used to stifle freedom of expression. For example, #Anonymous hackers created a "mashup" video of Tom Cruise effusively praising Scientology. Scientologists exploited copyright law in order to take the video off the Internet in order to suppress legitimate criticism.

A simple statement of "rights" would do much to clarify things. Today, the SOPA law (designed to protect copyright) is not unconstitutional. Now consider a "right" that says "Government shall not abridge access to the Internet". Suddenly, this proposed SOPA law is obviously wrong, because "abridging access to the Internet" is precisely what it does. It's not just copyright abuse, but issues from cyberwar to cyberbullying to regulation of Terms of Service to privacy: a clarification of rights is important.

Or consider phrasing it as "Government shall not abridge access to information". It's a minor change, removing the reference to technology. It introduces a new right, "information", that we didn't know we needed until the Internet came along (much how the printing press introduced rights we didn't know we needed). Information is every much as important as speech. But referencing the technology is easier: it gives us this new right, as well as resolves the complications with existing rights. If Vint Cerf followers convinced me I'm wrong, and that Internet (or cyberspace) is not a right, I would still insist that unrestricted access to information is a fundamental right that needs to be enumerated.

We can measure the importance of Internet-as-right in the inverse, in proportion to the efforts repressive governments take to restrict access to the Internet. Take China, for example. Their "Great Firewall of China" blocks large parts of the Internet. They force Google to remove items from its search results, such as any mention of the Tienanmen Square uprising, or even references to the recent Arab Spring (in case it's citizens get any wrong ideas). It's not speech being repressed here so much as access to information. In both repressive and free countries, we now see more attacks on Internet access than we do on speech, religion, or newspapers.

Vint Cerf is correct in saying that we need no clarification to know that the Egyptian Internet cutoff (to silence protests) was evil. Be he is incorrect in saying no clarification is needed. The First Amendment is not technology neutral, the 18th century version calls out the technology of the printing press, and the 21st century version should call out the technology of the Internet. Internet access is a human right, and even well-meaning governments are already infringing it, because of the lack of clarification.

Passwords: uniqueness, not complexity
Wed, 04 Jan 2012 18:32:00 +0000

Hacktivists recently broke into the StratFor website and dumped details of 800,000 accounts, including e-mail addresses and password-hashes. Since the password-hashes were simple MD5, it meant that almost all the passwords were easily cracked. People have looked at the passwords, and found that most people chose simple ones, such as "password123". This has led to articles like this one (Breach shows that even experts chose bad passwords) that claims "Security experts recommend building long, complex, case-sensitive passwords with multiple characters".

Nope. That's wrong advice. Your password for a free or cheap StratFor account doesn't need to be complex, because there is little to lose if hackers guess it.

Instead, what's important is that the password be unique. Most sites are like StratFor and have poor cybersecurity. (StratFor wasn't even close to good cybersecurity, they were horrible on almost any measure). Any information you give them, such as your password, will eventually get stolen by hackers. If you use the same password for all websites, then eventually hackers will break into one of those sites, then gain access to all your other accounts.

There are essentially three tiers of websites. At the first tier is your e-mail account. Since a hack of your e-mail account means hackers can reset passwords on all your other accounts, it would be terrible if that password were lost. This should both be very complex, as well as wholly unrelated to any other accounts.

At the second tier are important e-commerce sites, like Amazon.com, NewEgg,com, Apple.com, and so on. The major sites are unlikely to be hacked. You could probably share the same password for all these accounts.

At the third tier are the unimportant accounts, like StratFor, where it wouldn't be catastrophic if your password were lost. Again, you could choose a third, simple password, like "passwd1234" for all these accounts. It'll probably get stolen within a year, but who really cares?

Thus, you really only need three passwords for each tier, so it's not too much trouble. However, even then, you might consider adding uniqueness. For example, on the last tier, you might use the domain name as your password, like "passwdStratfor1". When a hacker breaks in and runs an automated script to see if your password is unique, the script will fail to find a match on any other site. Sure, a hacker looking at the password individually will figure out your scheme, but in a huge hack like the 800,000 StratFor accounts, hackers are unlikely to manually check every password.

In conclusion, your first password policy shouldn't be complexity, but uniqueness. When hackers break into a site like StratFor and discover your password is "password1", you shouldn't be embarrassed. You should instead say you don't care about your free StratFor account, or that hackers break into it, and that knowing this password doesn't help break into any account you do care about.

---

Updates:

Somebody also suggested Stiennon's article on Forbes Fallout from the Christmas Hack of Stratfor. His analysis is wholly incorrect == unless Stiennon has also tested those passwords to see if they were reused.

Rob Lemos criticizes password reuse at InfoWorld New year, same old security passwords

XKCD has an evil plan at https://www.xkcd.com/792/.

Nick Selby writes about Blaming The Victim in the STRATFOR Hack, how we need to stop blaming the people whose passwords were revealed, and start blaming StratFor for it's incredibad cybersec.

Predictions for 2012
Sun, 01 Jan 2012 18:09:00 +0000

We predict there is a more than 80% chance the Mayan calender is wrong and the world will not end. Other predictions we have are:

Cloud

Cloud cloud cloud cloud cloud. Whatever products/services people come out with in the next year, they will position them as being perfect (or even necessary) for the cloud.

SCADA/ICS

How many cybersec experts does it take to change a lightbulb? Yes, SCADA/ICS systems are 15 years behind in terms of security, and yes, there is usually a path that can be found from the Internet to these systems, but no, there is no huge danger looming on the horizon. There will be no massive power blackout in 2012, and nobody will die from a probably malicious attack.

Cyber-war

The cyber-military industrial complex still needs more funding. Congress will pass more laws helping them.

Hacktivism

#Anonymous #LulzSec #AntiSec #OhMy

We'll see more lulz, but no import hacks will happen, like exposing the cyber-military industrial complex that created Stuxnet.

Easy instructions for boycotting GoDaddy over SOPA
Thu, 22 Dec 2011 23:15:00 +0000

SOPA is a horrible internet regulation law pushed by the copyright cartels that will destroy many of the freedoms on the Internet, such as the TOR project that anonymizes network traffic for activists in repressive countries.

Go Daddy supports SOPA. Therefore, if you care about Internet freedoms, you should probably move your accounts to another registrar. This link http://blog.jeffepstein.me/post/14629857835/a-step-by-step-guide-to-transfer-domains-out-of-godaddy describes how to do it in a painless manner. I'm moving my Go Daddy registrations to Network Solutions, where I already have an account.

Freakonomics vs Cybersecurity
Fri, 09 Dec 2011 06:18:00 +0000

I saw this go across my Twitter feed, so I thought I'd write up a quick response. The cybersecurity view of economics is not the same as the economists view of economics. Using freaky economics like Freaknomics is a good way of explaining normal economics.

SpireSec Pete Lindstrom

Security freakonomics talk tomorrow... what should i say? ;-)

The first misconception of economics cybersecurity people have is calculating where the money goes, or how much things cost. That's "business", not "economics". If you are thinking in terms of "Return on Investment" (ROI), then it's not "Economics".

The second, and more common use of economics (in the field of cybersecurity), is the political attempt to prove that there is some sort of "externality" or "market failure" that means we get to punish Microsoft for its vulnerabilities. While the conclusion is faulty, this is a real economics concept. It describes the situation where I sell you fireworks, then you set them off, causing your neighbor's house to catch fire. The "failure" is that it's neither you (the buyer) or me (the seller) who paid the costs, but your neighbor. The cost of fire is an "externality", external to the original transaction.

The cybersecurity version is that when buyers buy Microsoft software, which has vulnerabilities, it's third parties who suffer. For example, a hacker might exploit a vulnerability in Windows, take control of thousands of desktops, and flood a website with traffic. That website suffers, even though it might not own any Microsoft products.

While this sounds plausibly "economic", it isn't. Consider the fireworks case. One solution to the problem is to fine the seller of fireworks, or regulate which fireworks they could sell. Another solution is to fine the customer who bought the fireworks and who lit them near their neighbors house.

Or, the third solution is punish the neighbor for having a flammable house.

Economics isn't about fairness, it's about the efficiency of results. It's that guy with the flammable, thatched roof that imposes costs on all his neighbors. It means the neighbors can't have a cozy fire in their fireplace during winter, they can't have BBQs in the summer, and they can't set of fireworks for celebrations. That is why local government usually choose the third option. They regulate how houses are built, and outlaw flammable roofs, believing this is the most efficient solution.

So which is the most efficient solution to Microsoft vulnerabilities? Blame Microsoft? Blame the user? Or blame the poor website victim? Or let the free market decide? I don't know the answer, but I know that I've never seen cybersecurity people make an "economic" answer based on efficiency, but instead, I've only seen arguments based on how Microsoft is big and evil, and how it's unfair to blame innocent users.


But this is just a tiny portion of economics, there is so much more. I recommend getting a college textbook on beginning economics, such as Greg Mankiw's Principles of Economics. Follow the link to the Amazon site, and you can read the first chapter for free, which outlines his basic 10 principles of economics.

Below, I take some of those basic principles and describe them in a cybersecurity context. Think of it as a useful way to learn economics if you already know cybersecurity, or as a way of learning cybersecurity if you already know economics.

The first principle from Mankiw's textbook is that cybersecurity is a tradeoff. In terms of logic, it's an XOR operator, not an AND. In terms of Heinlein (sci-fi author), it's TANSTAAFL - Their Ain't No Such Thing As A Free Lunch. Making the network more secure means making it worse in some other fashion, such as slower, less reliable, less user friendly. When cybersecurity experts say dumb things, there's usually a failure to acknowledge tradeoffs involved, that you must give up something in return for more security. The tradeoffs are not just between security and other things, but between two security choices. The funniest joke in cybersecurity are the two Wikipedia articles on Defense in Depth and Defense in Depth (computing). The original meaning was about trading off border security for better internal security, such as moving the troops from the border of a country to deeper inside. But no cybersecurity professional can admit to such tradeoffs, that it's ok to reduce security in some place in order to improve security somewhere else. So "defense in depth" has morphed into an argument that no matter how much security you have now, you need even more, both on the border AND in depth.

The second Mankiw principle is opportunity cost, or that the cost of something is what you give up to achieve it. The cost of cybersecurity isn't the money you spend, but what you gave up. Hiring another cybersecurity expert on your team means not hiring a saleperson who could sell more of your company's products/services. When you go to your boss and explain why your budget for cybersecurity needs to increase, you need to explain why the budget for marketing, sales, and RnD needs to decrease. During the dot-com era, companies that put up insecure websites first won the dominant market share, those that waited until their websites were secure lost. The opportunity costs of waiting until something is completely secure can mean your entire business.

The third principle is that rational people think at the margin. Cybersecurity people talk in absolutes, as if something is insecure or secure. They should instead talk in relative terms of "more secure" or "less secure". Moreover, they need to compare the marginal benefits in security to the marginal costs. That fancy new expensive firewall still won't make you secure, the question instead is whether the marginal improvement in security is worth the price over a cheap firewall. Or, take the TSA screening requiring people to take off their shoes. Cybersecurity experts complain that this makes no difference. They are wrong; taking off the shoes at security makes people marginally safer. The only question is whether this tiny improvement in safety is worth the enormous additional cost (probably not). Part of this is realizing that security has decreasing margin returns. The reason that Microsoft can't fix all their bugs is that the more bugs they fix, the more it costs to fix more bugs. Spending a million dollars might fix a 1000 vulnerabilities, but spending another million might fix only an additional 100 vulnerabilities. Spending a third million might fix only an additional 10 vulnerabilities. Spending yet another million might find and fix only one additional vulnerability.

The fourth principle is that people respond to incentives, perversely. A straightforward example is that of complicated password policies, the more complicated they are, the more a person is likely to write down the password on a sticky note underneath their keyboard, thus making the system less secure, not more so. The consequence of this is that people have a fixed risk tolerance. When you make things safer, people behave more recklessly. If you install anti-virus on their desktop, they are more likely to run e-mail attachments. Measured one way, such as on an obstacle course, talking on a mobile phone impairs a person's ability to drive. Measured with economics, we find that while people are on the phone, they slow down and otherwise drive more safely, to accommodate the distraction. Drivers slow down and pay attention when it rains to compensate for the additional danger, which means they speed up and drive more recklessly when the roads dry up to compensate for the increase safety.

Another principle is that the value of security isn't infinite. One of the fun things freaky economists like to do is calculate what a person's life is worth. For example, let's say that you put your kid in the car to drive to the store rather than paying the neighbor to babysit for an hour for $10. Dying in a car accident is the leading cause of death for children, and those deaths are overwhelmingly near the home. If the chance of death on that trip is 1-in-a-million, and you could've spent $10 to avoid it, this means you value your kid's life at $10-million. (Well, not, not exactly, I'm glossing over the fine bits to make a point). The same is true of cybersecurity, where people treat security as infinitely worth. That's why they can't deal with marginal benefits vs marginal costs: the marginal benefits of increased security are always infinite, according to cybersecurity experts. Given free reign, cybersecurity experts will make the costs infinite, too. The only way to satisfy them completely would be to turn off the Internet.

The sixth principle on Mankiw's list is that free-markets are usually the best, tempered by the seventh principle that sometimes government can improve on free-market outcomes (such as when there is a market failure and externalities). A wrong application of this principle was President Bush's "Strategy to Secure Cyberspace" that had the fatuous statement "federal regulation will not become a primary means of securing cyberspace ... the market itself is expected to provide the major impetus to improve cybersecurity". This is wrong because the free-market will never "secure cyberspace". Instead, the free-market is what determines how valuable cybersecurity is in the first place, identifying the truth that people don't want the tradeoffs needed to make the Internet more secure. I once gave a talk where I asked "Raise your hand if cybersecurity is your highest priority" (everyone: yes), then "Raise your hand if you use wifi" (everyone: yes), then "Raise your hand if you think your wifi is secure" (everyone: no). In other words, people claimed to want security, but even though wifi wasn't secure, they used it anyway. That's because people lie; they claim security has infinite importance, but behave as if it's a tradeoff. The free-market captures this true value, government regulation doesn't. When government starts regulating cybersecurity, we'll start complaining about it in much the same way we complain about the TSA and the Patriot Act (which make what many consider unacceptable tradeoffs for small marginal improvements in security). In many cases, the cost of "compliancy", proving to the government that you are secure, is starting to outweigh the costs of the actual security.


I could spend days talking about the freakiness of economics, and cybersecurity, but this gives you a taste.

---

I get more comments via twitter than the desired comments page. A particularly cogent one is:

Quick review of the Kindle Fire
Wed, 16 Nov 2011 02:01:00 +0000

The Kindle Fire is a $200 device, compared to $80 for the cheap black-and-white Kindle, or compared to $500 for the iPad.

The reason you want it more than the standard "eInk" based Kindle is that it can play videos, run apps, and show books in color (like comic books). The downside is that it weighs twice as much as the standard Kindle, lasts only 10 hours on battery (vs 30 days for the standard Kindle), and only works with WiFi (no mobile phone connection).

It's a 7 inch screen (14.6 ounces0 compared to the iPad's 10 inch screen (21.3 ounces). This makes the Fire better for traveling than the iPad. But, since I travel with my iPhone and MacBook Air, I really don't need another device to watch video. What I need is something that won't run out of batteries. For that reason, I'm going to travel with the eInk Kindle Touch (7 ounces), not the Kindle Fire. (In addition, the Kindle Touch as a mobile phone connection that can be used in emergencies when no WiFi is available, unlike the Fire).

There are plenty of annoyances with device. Some content ("The Watchman" and "The Economist") is shrunk and can't easily be expanded (The Watchman not at all, The Economist with zoom-and-pan for every page). The same thing happens to web-pages: the iPad and standard Kindle have been around long enough for people to format content for them, but they treat the Kindle Fire as a full-sized desktop screen, and not the 7 inch screen that it really has.

The web-browsing goes through Amazon's cloud for "acceleration". Instead of browsing the web directly, you go through Amazon's servers, which strip out all the stuff that slow down web browsing. In practice, I don't think this works so well. Yes, some web pages are "snappier", but at the same time, some things behave oddly. I'd have to spend more time at it, but I think my experience on the iPad is better.

But for all such annoyances, it works much like the iPad. As his biography points out, Steve Jobs was really angry at how much Android (which runs the Kindle) copied most of what makes the iPhone/iPad cool. It even does a couple things better, such as cloud integration. Your books, magazines, music, videos, and apps have two selections, those in the Cloud, and those on the Device (demonstrated in the picture above), and downloading from one to the other is simply the touch of a button.

Here's the verdict: it's not as polished as the iPad. Your 2-year-old or grandparent can't pick it up and immediately start using it, like she can with the iPad. But that probably doesn't matter, since 99% of the time is spent with the content (reading, watching, playing) rather than with the device software. You have to ask yourself if an extra $300 is worth the difference in usability/polish for the small amount of time between watching videos, reading books, or playing games. It probably is important for the very young or very old, or the very geeky, but probably not so important for everyone else.

Sure, the recipient is going to be disappointed you didn't pony up the extra bucks for an iPad, but otherwise, the Kindle Fire is going to make a great gift for Xmas. I think they will spend more time using it in the coming year than almost any other present you could give them.

---

I'm not part of Amazon's affiliate program. Following the links on this page to Amazon's site gives no benefit to me (which you can verify with View Source and check out the links). I thought I'd point this out since Amazon's evil affiliate program leads to so much spam. These are my honest opinions.

---

Here is a Kindle Fire review from The Daily. It agrees with the points I made. But whereas it stresses "it doesn't do anything exceptional well", the real point to remember is that it does everything adequately well. And, that's only during the 1% of the time you aren't watching a movie or reading a book, at which point, the experience is roughly the same.

Cloud Computing is a Commodity, not a Utility
Tue, 25 Oct 2011 02:39:00 +0000

A computing pioneer named John McCarthy (creator of the LISP language) died recently. Some are giving him credit for coming up with the idea of cloud computing in 1961, when he described "time-sharing" (the sharing of a mainframe computer) as becoming a "public utility" like electricity or water. Nothing can be further from the truth. Today's cloud computing is the opposite of the "utility computing" that he imagined. This is as absurd as saying the original Star Trek TV show, with a voice activated computer, somehow invented Siri, Apple's new voice response system for the iPhone.

The archetype of cloud computing is Amazon EC2, a network of thousands of machines that can do anything from run supercomputer simulations to serve web pages. But cloud computing is more than EC2. It's a whole range of things, from Google apps, to Apple's iCloud storage of music, to the Amazon Kindle's storage of books. I have over 300 books on my Kindle – not actually on the device, but in my account with Amazon. When I broke my Kindle by dropping it two stories, I bought a new one, and all my books were still there. That is cloud.

Sure, Amazon's EC2 sounds a lot like the time-share systems of the early 1960s, but there are important differences. The biggest difference is "how we got here". There wasn't a slow progression of huge "mainframe" computers, but a rapid change from mainframes to "personal computers" in our homes (following Moore's Law).

If other utilities had progressed at the same speed as computing, then we'd all have a small fusion reactor in our homes supplying our electricity. Your iPhone can supply, through the cell network, all of the time-sharing needs of the 1960s.

Yes, Amazon sells compute power, but the word you are looking for is not "utility" but "commodity". Here is the current Wikipedia definition of a public utility:

A public utility (usually just utility) is an organization that maintains the infrastructure for a public service (often also providing a service using that infrastructure). Public utilities are subject to forms of public control and regulation ranging from local community-based groups to state-wide government monopolies. Common arguments in favor of regulation include the desire to control market power, facilitate competition, promote investment or system expansion, or stabilize markets. … The term utilities can also refer to the set of services provided by these organizations consumed by the public: electricity, natural gas, waterand sewage.

None of the above applies to Amazon EC2. But, the following Wikipedia page on commodities sounds a lot like Amazon EC2:

It is used to describe a class of goods for which there is demand, but which is supplied without qualitative differentiation across a market. A commodity has full or partial fungibility; that is, the market treats it as equivalent or nearly so no matter who produces it. Petroleum and copper are examples of such commodities.

In other words, cloud computing is fungible commodity like oil and copper, not a utility like electricity or sewage.

From this perspective, recently deceased Dennis Ritchie (who developed C and co-developed Unix) deserves much more credit than John McCarthy. The reason Ritchie developed Unix was precisely to break the "utility" model of time-share computing up to that point, and to make computers into a "commodity". Today's cloud computers like Amazon EC2 run mostly Unix, and mostly code written in C. They run almost no code written in John McCarthy's LISP.

The personal computing and Internet revolution is a genie that escaped the "utility" bottle. Many want to put that genie back again, and regulate the Internet and computers like utilities. Their arguments always sound good, but they are deceptive. It's the old phone utilities that lobby for regulations requiring new VoIP companies to provide 911/emergency services, making VoIP much more expensive. Likewise, it's law enforcement that lobbied for laws requiring mobile phones to have GPS location tracking features again for 911/emergency calls, but which law enforcement also uses to locate criminals.

Another example is "reliability". We all get frustrated when computers fail (as BlackBerry users recently experienced). Regulators promise to improve reliability. But this comes at a cost. Reliability has decreasing marginal returns, costs quickly explode as government demands more reliability. Right now, bandwidth and cloud computing is free, but it means that sometimes when I try to sync my Kindle, it might fail for a few hours. Despite what the regulators promise, there is no such thing as a free lunch, and regulations will not simultaneously keep costs down and reliability up.

How we define the "cloud" means a lot for our future. We are putting more and more of our "stuff" in the cloud, which special interests want to regulate, control, and monitor. We will lose our freedom unless we fight to keep it. Unless we fight to keep the cloud a "commodity", it will indeed start to look like an Orwellian "utility".

---

The full quote from McCarthy speaking at the MIT Centennial in 1961:

"If computers of the kind I have advocated become the computers of the future, then computing may someday be organized as a public utility just as the telephone system is a public utility... The computer utility could become the basis of a new and important industry."

iPhone 4S for Sprint performance below par
Wed, 19 Oct 2011 00:20:00 +0000

I'm returning my Sprint version of the iPhone 4S. The performance is far below either AT&T or Verizon.

Everyone loves to hate AT&T, but here's the thing: they've invested massively in building our their network to support devices like the iPhone and Kindle. Even in place where AT&T famously struggles (New York City and San Francisco), I can usually get a fair data connection. In my travels, AT&T 3G outperforms Verizon 3G in almost every case.

Sprint hasn't made that investment. At least, not here in Atlanta.
Traveling around the Atlanta area at various times in the day, I find
that Sprint consistently lags AT&T for 3G speeds. Sprint struggles to
reach 1-mbps, while AT&T rarely goes below 3-mbps. At times, Sprint
goes down to dialup speeds. Here are example of back-to-back speed
tests. I have a lot more samples, but these are representative of what
I see.

To be fair, Sprint has invested a lot in a 4G technology (based on WiMax)
that works much better, where 7-mbps is common. But the iPhone
doesn't support 4G technologies like WiMax. In addition, many people
are finding that WiMax falls behind LTE (used by AT&T, Verizon, and
T-Mobile).

Sprint does have one advantage. Currently, it supports "unlimited"
downloads, whereas Verizon offers a plan with up to 12-gigs of
downloads, whereas AT&T offers at most 4-gigs of downloads. In
addition, it's one of the cheaper plans, with unlimited texting and
data for only $80/month.

But in practice, few people use more than 2-gigs per month. I rarely
do. In those cases, AT&T can offer cheaper and faster plans.
I don't know which will be best for you, but for myself, I'm choosing
an AT&T iPhone 4S and returning my Sprint iPhone 4S.

UPDATE:
These are the last ones taken this morning, AT&T with 5.07mbps and Sprint with 0.21-mbps:

Update 2:
I returned The Sprint iPhone today and was told I was the 18th person to do so for the same reason. This makes me sad as I was a big fan of Sprint rolling out the first nextgen speed network.

DefCon speakers guide to #OccupyWallStreet hand signals
Fri, 14 Oct 2011 23:42:00 +0000

Next year, at DefCon (the world's largest hacking conference), speakers are going to be confronted by hand signals (like 'twinkles') that were developed during the #OccupyWallStreet protests. That's because much of the audience will have attended one of the many "Occupy" protests.

So that speakers don't get weirded out by this, I thought I'd write up a brief guide.

TWINKLES

This is the most common hand signal. The audience raises their arms and flutters or "twinkles" their fingers (also known as 'spirit fingers'). This means they like what you are saying. It is a silent form of clapping, so that they do not interrupt you.

Down-twinkles (arms up, but fingers pointed down) indicates the reverse, displeasure at what you are saying. You'll get a lot of these when you talk about how good Windows security is, for example.

CROSSED ARMS/BLOCK

When they hold their arms in front, crossed, it means they dislike what you are saying so much that they are about to get up and leave.

WRAP IT UP / GET TO THE POINT

There are two forms of this. One is the rolling motion, holding the arms in front and rolling them around each other. The other is holding the finger up and making a circling halo motion around the head.

It means the audience things you are rambling, and wants you to get to the point.

When you go beyond your allotted time, these will break out in the audience, indicating your should wrap up your speech and leave the stage for the next guy.

PLEASE CLARIFY

In this signal, the audience member holds up an index finger. There are some variations, such as holding up two bent fingers, or cupping the hand into a "C".

This asks you to clarify the point you just made, such as providing more details.

It's also done when they didn't hear you for some reason. For example, let's say there was a loud bang somewhere, the audience will hold their hands up like this, so just repeat what you just said. If it's an issue where they have problems hearing you general, they will do EARS.

EARS

Holds the palm or cupped hand to the ear. Means they can't hear you. Pretty obvious.

DIRECT RESPONSE

In this signal, the audience uses index fingers (or full palms) going back and forth. Both hands are used, with each hand going the opposite direction from the other.

This means the audience member has something to say, to correct something you've said, or to ask a clarifying question.

PROCESS POINT/OFF-TOPIC

This is where they make a tent above their head with their fingers. It means you are getting off-topic.

THUMBS UP, DOWN, TO THE SIDE

This is how the General Assembly at the Occupy movement votes on things, with the obvious result that thumbs-up means yes, thumbs-down means no, and thumbs-to-the-side means abstain. I'm not sure why you'd see these while speaking, but I thought I'd mention them.

---

Here's a video that explains some of these:

Scanning the Internet
Tue, 11 Oct 2011 20:33:00 +0000

As part of a research project we are port scanning the entire internet. The scans will come from 216.75.60.94.

EDIT: Per a comment I realized I left alot of stuff out. Here ya go:
I am scanning everything from 1.0.0.1 to 223.255.255.255.
I am collecting hostname, IP address, OS type, and service version.
As far as how long I have no idea, I am guessing somewhere around 100 days.

I am aware Shodan offers this information now, I need to collect my own data for this project however.

EDIT: This isn't a big deal. Researchers like us frequently scan the IPv4 address space. At any point in time, there are a few "white-hat" researchers doing such scans (we know of one other group currently conducting a scan), and many more "black-hats" doing it. The reason for this post is simply to be on record about it.

How geeky am I
Sun, 09 Oct 2011 06:20:00 +0000

These days, we have both 1080p hidef television, and plotlines with hackers. That means "code" appears frequently on the screen. Of course, if you read the code, it has nothing to do with the plot. The producers just grabbed a fragment off the net and stuck it in there for dramatization.

In the pilot episode of the remake of Charlie's Angels, one of the gals cracks an electronic safe by typing in a fragment of code. Here is a picture from the video:

To a geek like me, two things jump out. The first is that this code includes a hard-coded random number generator (i=1103515245*i+12345&0x7fffffff)/2147483648.0). That's odd, why code your own instead of using the built-in random number generator known as "rand()"? That leads to the second observation: the code is deliberately obfuscated, possibly because it was an entry to the (now defunct) Obfuscated C contest.

Because of the obfuscation, it was difficult googling it, but I eventually found the source at this PasteBin link http://pastebin.com/ETeBXXGh. I copied it and listed it below. It's the only version of the code I could find. From the additional code, it appears probable that it's a generator/solver for Sudoku. I tried to run it, and didn't get any good results, because I don't know the data it expects. I could figure it out, but ...

...now I'm bored with it. I found the code, I know it's something to do with Sudoku, that it's obfuscated, but I no longer care to find out the rest.

---

Update: Bah! A malware analyst "ocean" found it https://twitter.com/#!/_ocean/status/122956436635529216 on the IOCCC website http://www.ioccc.org/2005/aidan/. It probably took him the extra 30 seconds I wasn't willing to spend, it's obvious.

---

Update: Check out Ian Eiloart's comment at the bottom. As he points out, Hollywood is a stickler for copyright. The Obfuscated C contest explicitly states that all submissions must be in the public domain. This makes it perfect for Hollywood: you get code fragments that both extremely geeky and for which you don't have to worry about a lawyer serving your papers. Note that there is a "fair use" clause that means they don't always have to worry anyway. For example, my use of the picture from the show above is covered under "fair use", so I can use it without having to ask for permission.

---

#include 
#include 

#define N(I,l) s l]=(I?1<#define f(a,t) for(a=0;a#define Su(d,o,ku) O(l/9,d) O(l%9,o) O(l%9/3+l/27*3,ku)
#define NO ;printf("%c %s",I?I|48:46,++l%3?"":l%9?"| ":l%27?"\n":l%'Q'?z:"\n");
#define Ba(k,a) {O||printf("!!! " #a " %i\n",k+1);goto l;}
#define O(o,k) f(l,9) c[l]= *#k?0x3fe:-1;\
  f(l,81) if(*#k){\
    if(!(s]&c[o])) Ba(o,k) c[o]&=~(1<<(s]>>10));\
  } else if((s]>>l0)&1) c[o]=c[o]+1?-2:l;\
  if(*#k) { f(l,81) if(s]>>10||(s]&=c[o]),!s]) Ba(o,k) }\
  else f(l,9) l[c]<0||s[c]]>>10||(N(l0,[c]),C++);

struct{ int s[81],I,l,O; } S[0123];
int I, l, l0, o, C, O=0, w=0, c[10], L; long i;
char z[] = "\007      &       &      \n";

#define s S].s[l

int main(int n,char**N) {
    S->O=0; L=n>1?*N[1]-85?1:6:0; i=L&1?atol(N[1]):123;
#define i (int)(81.0*(i=1103515245*i+12345&0x7fffffff)/2147483648.0)
    for(l=C=0;l<81;) {
 I=L&1?0:getchar()^48; i; I=I-30?I:0;
 if(I<10) {
  #define S S[O
  if(C<22) z[C++]^=13; N(I,)NO
 }
    }

    for(;;) {
 l0:
        Su(row,col,box) C=l0=0;
 f(l,81) if(!(s]>>10&&++l0)) {
   o=s]&1022; for(I=0;~o&1&&(o/=2);I++); o-1||(s]|=I<<10,c++);
 } if(l0==l) {
   if(O&&L&2) { O--; goto l0; } goto O; 
 } for(l0=1;10>l0;l0++) { Su(,,) }

 if(!C) {
     l=(o=S].O)?S].I:0; I=o?S].l%9+1:(S].O=i%9+1);
     for(;l<81;l++,i=s].o,o=0) if(!(s]>>10)) {
  for(;;I=I%9+1,o=1) {
      l0=0; if(o&&I==S].O) goto O;
      if(s]>>I&1) {
   S].l=I; S++].I=l; S]=S-1];
   N(I,); O>w&&(w=O); goto lO;
      }
  }
     }
 }
 lO: S].O=0; goto l0; l: if(!(l0=O)) { L=0; goto O; } O--;
 s,S].I] &=~ (1<    }

    O: switch(L) {
 case 2: if(l0) {
 case 3: O=82;
     for(S].l=l=i%81;l0||S].l-l;l=(l+1)%81)
           O1: if(l0=0,S].O=s]>>10,~s]&1) {
  s]=1023; S].I=l; S&0]=S]; w=O=0; L=2; goto lO;
     }
     L=0; l0=1;
 } else {
     l=S=82].I; N(S].O,)|1; goto O1;
 }
    }

    printf("\n\n"); for(l=0;l<81;) {I=s]>>10 NO}
    printf("\n%s (stk %i %i)\n",l0?"Done":"No way!",O,w);
    L|=2; if(L-3||!l0) return!l0; S,82]=S]; goto O;
}

If you run it, and pass the source as the input "a.out foo.c", you get the following output:

. . 1 | 1 . . | 1 7 7
6 . 9 | 9 9 3 | 2 7 3
4 8 4 | 6 3 9 | 2 7 1
------+-------+------
9 . 3 | 1 8 1 | 1 1 .
. 1 1 | 2 8 1 | 1 . 9
. 1 . | . 8 1 | . 1 2
------+-------+------
3 . . | . 1 . | . . 7
. . 1 | 1 8 5 | 1 6 .
1 1 1 | 2 3 8 | 1 . .
!!! row 1


. . 1 | 1 . . | 1 7 7
6 . 9 | 9 9 3 | 2 7 3
4 8 4 | 6 3 9 | 2 7 1
------+-------+------
9 . 3 | 1 8 1 | 1 1 .
. 1 1 | 2 8 1 | 1 . 9
. 1 . | . 8 1 | . 1 2
------+-------+------
3 . . | . 1 . | . . 7
. . 1 | 1 8 5 | 1 6 .
1 1 1 | 2 3 8 | 1 . .

No way! (stk 0 0)

Independent reporting of #OccupyWallStreet
Thu, 06 Oct 2011 21:02:00 +0000

I was unhappy with the poor journalistic coverage of the #OccupyWallStreet protests, so I went to Wall Street myself to see what’s going on, and report on it.

It’s the quality of the coverage, not the amount that's the problem. It’s been on the nightly news every night for the past week, but there has been little "serious" reporting.

By "serious" reporting, I mean such things as contacting the park’s owners asking for an official statement. The protesters are occupying Zuccotti Park, owned by the same company (Brookfield Office Properties NYSE:BPO) that owns the adjacent skyscraper. An obvious step would be to contact them asking for a statement, but I could find no journalists that had yet done so. Well, if "journalists" aren’t going to do this, I can do this myself. I sent an email to their VP of Communications. I got a response, which I posted to my blog. When I posted it, I also Googled the sentences from the official statement, and found no results. I was indeed the first one "reporting" on this. Since then, others have mentioned the official statement, probably by picking it up from the #OccupyWallStreet Twitter hashtag that links to my blog.

Brookfield's official statement expressed their frustration with how the protesters were breaking the rules of the park (my blog post shows a picture I took of the posted rules). In particular, they haven’t been able to do their daily maintenance and cleaning of the park for the past three weeks. For a reporter, that leads to the obvious question: is the park staying clean? and if so, how? The answer reporters would find is this: the protesters themselves are taking care of this. They are exhorting people to not litter, they are making sure the trash cans have fresh bags and patrol the park picking up litter. They make sure the trash bags are set out in the right place to be picked up by the city’s garbage service

If I were a reporter, I would then follow this thread: The protest started as a chaotic event put together haphazardly via Twitter and the Internet, with no actual leader. How, then, were they able to organize a garbage detail? The answer is self-organization. Protestors have developed a General Assembly of all the people that gives authority to the "Central Committee," made up from the hard-core protesters who are sleeping in the park night after night. The Central Committee has many subcommittees, like the "Media Team" responsible for recording the proceedings or the "Arts and Culture Committee", responsible for making signs and running the drum circle, and the "Sanitation Committee" team keeping the park clean. They have organized the park into specific areas, dedicated to different tasks.

Let’s follow this thread even further. The protesters aren’t allowed to have a bullhorn or loudspeaker. How, then, can a person address the General Assembly, in the middle of a bustling city, reaching the hundreds of protesters spread throughout the park? The answer is the "People’s Microphone". A speaker speaks in short phrases. Those nearby then repeat the phrases, shouting so that those in back can hear. The People’s Mic is powerfully emotional, driving home the point of solidarity. Although, it’s occasionally ironic when a speaker says things like "we are all individuals" or "we must think for ourselves".

More than just the amplifying the voice, there is a system for selecting speakers. There is a "Stack" of speakers expressing desire to speak, with their position on the stack dynamically adjusted so that all points of view get equal time, or so that shy women get pushed ahead in the stack to counterbalance loud males. The audience gives feedback, from up/down thumbs, to raised hands with wiggling fingers ("twinkles") to express enthusiastic support, like clapping, but without drowning out the speaker with noise. (Apparently, this structure was inspired by the Spain "Indignados" protests from back in May.)

This organization is visible on the live streaming video and other efforts the Media Team has used to exploit social media to their cause. Inspired by the New York occupation, other groups in most major cities have already started their own occupations, or plan to do so soon. In my own Atlanta, they plan for this coming Friday. These new occupations share the same organization, e.g. the General Assembly, the People’s Microphone. When somebody writes the definitive book on this, I’m sure this organization model will become a blueprint for protests years from now.

As time has gone on, established liberal/progressive organizations have lent their support to the occupiers. The crude hand-made signs from the first couple weeks are giving way to slick printed placards. The question is, as time goes on, will the movement be lead by the hard-core who slept night after night on the cold hard ground and who have worked to create their own organization, or will it cede control to established political operatives? As we saw with the Tea Party, a grass roots effort was quickly hijacked by skilled politicos.

The point I’ve been trying to make with the last few paragraphs is that there is a "story" here. I started with the obvious task of asking the owner of the park for an official statement about the occupiers of the last three weeks, and following those threads, I saw a story emerge that is different than the standard narrative of "just-another-protest".


There are many other aspects of this that go unreported. One I find especially important is the loving nature of the protest. If you look at photographs in the news, you see the typical angry protester. This is the sort of action shot newsrooms prefer, i.e., showing the emotion of the scene.

But the protest isn’t angry. Quite the opposite, it is loving and accepting. If you go up to protesters with the opposite political view and debate them, they will express their undying love for you and ask for you to join them to increase the diversity of viewpoints. I did this myself, and watched this happen to others, including cops. This attitude pervades everything they do, and is frequently reinforced by the hard-core occupiers.

This is the opposite of what happened during the protests against the Iraq war, the protests against the last Republican convention in New York or the violent protests during every G8 summit. Not only is this different than most other protests, it is the similar to the hyper-tolerant "Burning Man" festival that takes place in the Nevada desert every summer. Whether it’s Burning Man or Occupy Wall Street, there is a cultural shift somewhere here. Now I feel compelled to go to Burning Man next year, just to track this thread down.

In many ways, the press treats this protest the way they treated the Tea Party, completely distorting the story. Journalists ignored the mainstream of the Tea Party and instead focused on the fringe. Instead of showing the hundreds of signs calling for smaller government, reporters instead focused on the one sign showing Obama as Hitler. In the end, this reporting became self-fulfilling. The Republican fringe disaffected with the establishment were convinced by this reporting, believing that they, too, should join the Tea Party, thus derailing it.

This is a particular danger to the Occupation movement. They still haven’t defined themselves, and risk letting the press define the movement for them. They started out with the idea that occupying Wall Street for weeks would be a good way to get their message out, but they are still trying to come to consensus on what, precisely, their message is. The press (and critics) claim they need a message and that they need a concrete list of demands, but I’m not sure that’s true. This is something else, something new, something that doesn’t need to be defined by the old.

In that way, it’s like the Internet. When the Internet appeared on the scene 20 years ago, it wasn’t like anything that predated it. Yes, you could define it in terms of the old, as a digital library, as an electronic form of mail, or as a communications network, but none of these descriptions captures the essence of what the Internet really is.

In particular, there is the problem with the "filter bubble". While the Internet can expand a person’s universe, it gives people the power to shrink it. People create a "filter bubble" around themselves, using tools of the Internet to pass only those things they agree with. For example, Google watches what people search for, profiling them, and sorts the results for that individual. They see their own small universe reflected back, rather than the big universe.

That’s why, despite appearing nightly in the news, the occupiers feel the press is ignoring them. This protest has become the most important thing in the world -- among the people in their filter bubble and those in their social network. It becomes difficult for them to imagine that this isn’t the most interesting thing to everyone else as well. They apparently don’t comprehend that the "news" just reflects what the organizations think their audience wants to hear. If the public doesn’t seem to care, neither does the press.

There is much more to this filter bubble. An obvious problem is that people filter out opposing political views. But they also filter out intellectual arguments that otherwise agree with them. They’ve filtered their view of the world so that political arguments are black-and-white, rather than grey. In their filtered view, politics is about propaganda and rhetoric, rather than debate.

I interviewed the hard-core protesters, those sleeping in the park overnight. I found only propaganda. They could repeat word-perfect the propaganda about the execution of Troy Davis case, but none of the details from the Wikipedia entry on the case. They could repeat the propaganda of Al Gore on Global Warming, but none of the science from the UN IPCC that declares the scientific consensus on the issue. They could repeat the economics of Michael Moore, but not that of Paul Krugman, Nobel laureate, writer of the popular liberal/progressive blog "Conscience of a Liberal" at the New York Times and author of a college textbook giving an introduction to economics. For example, the protesters say "the rich get richer but the poor get poorer," whereas Krugman says "the rich get richer but the poor go nowhere". This is due to a profound disagreement about a basic economic concept and the economic data.

As the protesters try to define themselves in order to come up with a coherent political platform, they are hindered by this filter bubble. The forces will drive them to come up with something that excites their small group, but which will prove unacceptable to the larger world. I think they have to learn to reach outside their bubble if they want to actually influence things and to become to the Democrat Party what the Tea Party is to the Republican Party.

I get the impression that the entire Occupy Wall Street movement needs a "[citation needed]" footnote. Wikipedia uses this technique to allow anybody to challenge an unsupported assertion. Anybody can insert this footnote, expressing to the reader that (as yet) the assertion isn’t supported. Anybody else can find supporting evidence, and replace the [citation needed] to a footnote pointing to a reliable source. If no citation can be found, the assertion is eventually deleted.

I’m concerned by the lack of scholarship because of the history of populism. The occupiers were inspired by the Arab Spring, where the people took their countries back from powerful dictators. But they forget that those dictators similarly took power at the head of populist movements that removed their predecessors and that they ruled "in the name of people". Colonel Gaddafi didn't promote himself to General because that was presumptuous, he was just a man of the people.

I found the occupiers had the same totalitarian attitude, though they don’t see it as totalitarian. Yes, their loving acceptance of those who disagree with them is astonishing, but it’s totalitarian. It asks that people give up their individuality to the state the occupiers are creating. Rather than free speech, the protest has a sort of "managed speech" to make sure everyone has equal time. There is also the flip side, that not to join the movement or to disagree with the protesters means that you are working against the interest of the people.

We have seen this before in history, such as during the French Revolution and the Reign of Terror. After they ran out of nobles, the Committee for Public Safety started beheading political rivals -- even those of their own party who helped overthrow the royalty. Their implicit thinking was this: I support the people. Therefore, if you disagree with me, you are acting against the people and must be beheaded. Or to paraphrase in the modern idiom, "you are either with us or against the people".

The protesters have been settling on the idea that the conflict is the 99% against the 1%. But since the country is evenly divided between Democrat and Republican, they represent, at best, the interests of 50% against the 1%. No matter how poor, Republicans don’t see socialism as being in their own interests. Instead of chanting "We are the 99%" they should be chanting "We are the 50%", but they seem immune to seeing things from this perspective.

I personally experienced this duality between populism and totalitarianism. I had chosen a table in an empty area away from the crowd to type up my notes. I didn’t realize it, but it was near the General Assembly area that would soon become crowded. Members of the Media Team came up to me and insisted I move, so that they could set up a tripod and camera on the table to take pictures of the General Assembly. I refused. I tried to do this as nicely as possible, with a pleasant demeanor, but of course, I was being a jerk. I didn't like they way they insisted, but also I wanted to test them, to see what would happen when somebody didn't go along with their demands.

Of the three people, one was nice. He smiled, shook my hand, and said "peace". I’ll bet he’s been to Burning Man. But the other two were nasty. The second guy, visibly twitching in anger, made unspecified threats that I had better move. The third person, tried to argue. She claimed that the protest had prior right to this spot, since they had been occupying the park for weeks (a fallacious argument, since the owners declare the park open to everyone equally). She then argued that this was for the entire group, to get the word out about the protest, to which I answered that I’m not part of the protest, that I don’t share their views. Her final argument was the totalitarian argument: this is for the people. She then proceeded to say that she was going to setup the tripod anyway, and that if I didn’t move, she would accidentally step on my laptop computer, because her attention would be on taking pictures and not where she was stepping.

Again, I admit to being a jerk here. But I’m a libertarian, which means I’m interested in the connection between populism and totalitarianism, which we libertarians see as the same thing. I wanted to experiment with it.


Back to reporting. I see it as a struggle between the "story" and some sort of "narrative". Take, for instance, the most reported event of the protest, the arrest of 700 protesters as they tried to cross the Brooklyn Bridge. However you treat the story, you have to struggle with the "narrative" that "police oppress protesters". Here’s what happened. The occupation is of the park in Wall Street. Last Saturday they marched from there intending to go to the park right on the other side of the Brooklyn Bridge, then back again. The march was planned ahead of time. The protest leaders talked to the police about it. The police told them to stay off the roadway to avoid blocking traffic, and instead use the pedestrian walkway one level above the roadway. The protest leaders widely communicated this to other protesters.

But at the same time, some protesters were hoping for a confrontation with the police, because mass arrests would get them on the news (I overheard two protesters discussing this). Others were passing out pamphlets on what to do when arrested and urging people to write the phone number of the National Lawyers Guild on their arm. Some of those arrested were among the Central Committee, who would have been the most likely to have known they should not have been on the roadway.

When the protest happened, many protesters followed the correct path above the roadway, but many others incorrectly chose the roadway. After about 700 had taken to the roadway, the police closed off both ends of the bridge, preventing them from escaping and arrested them all. Eventually the errant protesters were given summons for causing a public disturbance. Protesters accuse the police of causing the problem by letting protesters out onto the roadway in the first place rather than informing them to take the pedestrian way. They also point out that shutting down the bridge for hours caused much more of a public disturbance than letting the protesters pass for 15 minutes. Regardless of any agents provocateurs on both sides, though, it’s a good bet that the bulk of the 700 who got arrested were just sheep, going along with the crowd.

For me, that’s the "narrative": stupidity and ignorance on both sides cause things like this, rather than malicious intent - barring a few on both sides who want to see the problem escalate.

The arrests themselves were interesting. The protesters above, on the pedestrian level, were not arrested, but shouted/chanted encouragement to those below. There was confusion about how to act during the arrests. Should they do so in the nice, polite, accepting manner that defines the niceness of the movement? Or should they act like traditional protesters, lock arms, and passively resist? They seemed to be split half and half. Again, I blame the media: protesters watch the news, and try to copy how they see protesters act, making the news retroactively correct.

That "the revolution will not be televised" is a famous political song describing the 1960s political movement. The 2011 alternative is a revolution on Flickr, Tumblr, and streamed live. Look at the photographs from the bridge arrests. Almost everyone, both among the protesters and the police, has either traditional cameras or smart phones. You see a standoff between the police and the protesters, with each side pointing cameras at the others. Pictures taken facing into the crowd show a sea of cameras facing right back. Every one of these cameras is connected to cyberspace. Some of them even use applications to send the pictures and videos live to the Internet, so they are preserved even if the police confiscate the cameras and delete the pictures.
The Internet is a force multiplier. There are actually only a couple hundred protesters sleeping night after night in the park. But their hard-core determination inspires a couple thousand during the day, 10,000 watching the live stream, and a 100,000 participating via social media.

In one incident, there was a traditional news team from Fox News, trying to do an interview. The interviewee went on a tirade against Fox News. Those within the filter bubble of the protest loved it, but of course, it’s perfectly useless to a news station reporting on the protest.What I found interesting about this incident was the claim by the Fox News reporter that the protesters can’t get their message out without the mainstream media. But that's false. The protesters are getting the message out via the Internet just fine. Indeed, neglect is preferable to the distortions as the media tries to pigeonhole the protest into their preferred narratives.

By the way, while Wall Street may be responsible for bad things, it is Wall Street who financed putting a million miles of fiber optic cables crisscrossing continents and under oceans. It is Wall Street that financed the thousands of cell towers. It is Wall Street from which venture capital comes to finance startups like Twitter. Thus, tweeting "Down with capitalism" from your iPhone for those around the word to read seems to be the most ironic thing a person can do. The live stream from the protest site, shared with 12,000 (at this moment) people across the Internet is a testament to Wall Street's allocation of capital that these protesters fight against. [Obligatory Monty Python reference]

That the protest is dominated by Internet savvy youths exploiting social media is frequently mentioned. But what is not mentioned is the fact that the protesters are overwhelmingly college students, or recent graduates who still haven’t found jobs. They aren’t just any college students, but the stereotypical sort that you might expect to be involved in campus activism, such as graduate students in "Gender Studies." I found nobody with engineering or science degrees, but many from arts and acting colleges. After talking with one guy for a while about unemployment and his difficult in finding a job after college, I found out that he was a "poet." I’m not sure he understood that employers aren’t looking to hire poets. The only person I met that had a political science degree was one of the police officers "keeping the peace."

The protesters are also predominantly white with blacks underrepresented. On the flip side, blacks are over-represented in the police force. The protesters often compare themselves to the Civil Rights Movement, but the photographs of the recent arrests often show black policemen arresting white protesters. I don’t know if this is a vindication of the Civil Rights Movement or if there is still more work to go, to get the blacks better ensconced in middle-class American to send their kids off to college with that combination of privilege and entitlement that turns them into protesters.

The makeup of the protesters also led to amusement among the cops, stationed in pairs on all four sides of the park. For some, their normal beat is in the poor areas of New York City. The police, who daily see the struggle of the real poor, had little use for protesters complaining about jobs while they carried around expensive MacBook computers paid for by their parents.

I mention the racial makeup for a specific reason. The Tea Party was also predominantly white, which was frequently reported in the news, despite the fact that guidelines tell reporters to avoid mentioning race when it’s not relevant. They nonetheless reported it because it fit the narrative they wanted to tell about the Tea Party (that it has a racist component). In much the same way, they don’t mention the racial makeup of the Occupation because it doesn’t fit their narrative.

Every night is like a blowout bash you organized in college. After everyone has gone home or passed out, you sit on the top of the dorm with close friends, too excited to sleep, but too tired to do anything else but sit around in small groups and chat. That’s the vibe from the park at 2 a.m.: Quiet hours started at 10 p.m., most everyone has left, many are now asleep over there in the sleeping areas, but many are still too excited to go to sleep themselves. They huddle together in intimate groups around the park, discussing things.

I think it’s the intimacy and restrained excitement at night that is part of the real story here, not the hubbub during the day that the press tries to mold into their narrative of just-another-protest. What makes this different are those protestors staying night after night in the park. Yet, news reporters flock the scene at 2 p.m., but are absent at 2 a.m. I can’t understand why somebody like the New York Times isn’t sending a reporter down there to embed themselves in the occupation, sleeping there for a week and perhaps writing a Pulitzer prize-winning story.



Conclusion


Here's my point: the press and pundits have already decided on the "narrative" that's independent of what's really going on. For example, many Republicans and Fox News commentators insist that this is "planned" by the left for some nefarious purpose. It isn't (although that might change if politicos seize control of the occupation). Conversely, the Left has a narrative about police oppression that isn't quite right, either.

I see a different narrative. The love and acceptance of dissenting views is huge. The intimacy of the occupation over night is amazing. The excitement from the live stream and Twitter feed is infectious. The populism hinting at totalitarianism is frightening. The occasional irony is amusing. More citations are needed.

I think there is something interesting going on here. It’s not just another protest. I think it’s a more enduring addition to our culture. A decade from now, when the U.S. invades France over a cheese dispute, protesters will "occupy" the streets using the same principles being developed now.

@Anonymous's war against the New York Stock Exchange
Wed, 05 Oct 2011 21:36:00 +0000

The hacker collective known as "Anonymous" (sic) has declared war on the New York Stock Exchange (NYSE), promising to "erase" it from the Internet this October 10th (in support of #OccupyWallStreet). Should we be afraid of this threat?

No. Hackers who can, do. Those who can't, make threats.

The most likely threat would be a massive DDoS attack, like that Anonymous did against PayPal. In that attack, they posted a program called LOIC on various forums. Activists downloaded it, ran it on their computers, which then flooded PayPal with traffic. That attack affected PayPal briefly, but at the same time, it left fingerprints behind identifying people running LOIC. The FBI followed up and arrested many of these activists. It's not something activists would be willing to do again on a large scale.

Unlike PayPal, the NYSE website is not the real NYSE. You can blow it up with explosives and you won't affect trading. Such a flood could "erase" it temporarily from the Internet, but everyone would yawn.

There are more practical things that could be done, but here's the thing. If you could do it, you could make billions of dollars.

For example, there are a lot of trader terminals connected more deeply with actual trading network, which is completely disconnected from the NYSE website and the Internet. Such a system could be subverted and cause minor disruptions with trades. Even major disruptions can quickly be fixed: simply shut down the exchange, fix the problem, and bring it back up again. 9/11 disabled NYSE, and it came back a few days later. I doubt there is a way to permanently "erase" it.

But if you could do that, you could do something better. If you weren't interested in making money, the thing to do wouldn't be to DoS the stock exchange, but let them DoS themselves. Corrupt trades in a way that's undetected for as long as possible. The various counterparties would then be locked up in lawsuits for the next decade.

---

So technically, how could a hacker get inside the network?

The NYSE runs a completely separate network. Well, lots of people say this, like the operators of the power grid, and it's rarely true. But it's true in the case of the NYSE: I doubt hackers will find a way from the Internet into the NYSE private network.

But, there are lots of things on the NYSE private network, such as terminals on the desks of traders among the members of the NYSE. If a hacker could get physical access to one of those terminals, he could do a lot of damage.

The backend computers aren't the sorts hackers have experience with. Instead, they are things like AS/400 from IBM or "nonstop himalaya servers" from HP. These are actually FULL of vulnerabilities. It's astonishing how weak they are. But nobody knows, because the vendors assure customers they are secure, no hackers have challenged this impression (because they can't afford $100,000 for a system to test with), and nobody really cares, because they think the network is secure from outsiders.

Thus, a good hacker, one who can reverse engineer and write custom shellcode, will find that the network is actually fairly open. But the casual script kiddies like Anonymous aren't likely to find success.

---

Update: It was a non-event, reported on here:

http://www.chicagotribune.com/business/breaking/chi-anonymous-takes-down-nysecom-for-1-minute-20111010,0,1627656.story

http://www.forbes.com/sites/chrisbarth/2011/10/10/blink-and-you-missed-it-anonymous-attacks-nyse/

Brookfield Properties responds re: #OccupyWallStreet
Tue, 04 Oct 2011 20:17:00 +0000

(For my complete report on the protest, click here.)


The #OccupyWallStreet protest is in fact occupying Zuccotti Park, a private park owned by Brookfield Office Properties. I couldn't find an official statement from them on the protest, so I sent an e-mail to their Communications department. This is the e-mail I got in response:

Robert,
Thanks for your note. Here is our statement:

As the owner of Zuccotti Park, Brookfield Office Properties is committed to maintaining a clean and safe environment for the public to enjoy.

For more than two weeks, protestors have been squatting in the park. Brookfield recognizes people's right to peaceful protest; however, we also have an obligation to ensure that the park remains safe, clean, and accessible to everyone.

Basic rules intended to keep the park safe, open, clean, and welcoming to all visitors are clearly posted. These rules include bans on the erection of tents or other structures, as well as the placement of tarps, sleeping bags or other coverings on the property. Lying down on benches, sitting areas or walkways is likewise prohibited. Unfortunately, many of the individuals currently occupying the grounds are ignoring these basic yet necessary requirements, which interferes with the use of the park by others, including local residents, office workers, and visitors.

Sanitation is a growing concern. Normally, the park is cleaned and inspected every weeknight. This process includes power washing, litter removal, landscaping and other maintenance as required. Because many of the protestors refuse to cooperate by adhering to the rules, the park has not been cleaned since Friday, September 16, and as a result, sanitary conditions have reached unacceptable levels.

We continue to work with the City of New York to address these conditions and restore the park to its intended purpose.

Best regards,
Melissa



Melissa Coley
Vice President, Investor Relations & Communications


Brookfield Global Real Estate
Brookfield Office Properties
Three World Financial Center
200 Vesey Street, New York, NY 10281-1021
T 212.417.7215, F 212.417.7272
melissa.coley@brookfield.com
www.brookfieldofficeproperties.com

The undersigned is an associated person of a registered investment adviser. View important disclosures and information about our e-mail policies http://www.brookfield.com/supervisedemaildisclaimer.

In my experience, sanitation isn't necessarily an issue. Unlike 'angry' protests that trash their venues, this one is (barring a few exceptions) very 'nice'. The protesters themselves are keeping the park clean. There is a strong ethos to not litter or otherwise degrade the park, and I watched as members of the protest went around the park with trash bags cleaning up litter. This doesn't solve the problem of hosing down the pavement every once and a while, but generally, the protesters are doing their own maintenance.

In this, and many other ways, the protest reminds me of the Burning Man festival held in the Nevada desert every summer. That, too, has an enormous social norm of keeping the desert clean, to leave the area as if the massive event didn't happen.

While complying with all the rules is difficult, since the purpose is to 'occupy' the park, I'm sure the protesters would accommodate Brookfield on other maintenance issues. Udate: As it turns out, the protesters refused all attempts to work with Brookfield to do things like hose down the park.

Above is a picture of the rules of the park. I took this picture at night, which is why it's colored yellow from the sodium vapor street lamps. In case you have trouble reading it:

ZUCCOTTI PARK IS A PRIVATELY-OWNED SPACE THAT IS DESIGNED AND INTENDED FOR USE AND ENJOYMENT BY THE GENERAL PUBLIC FOR PASSIVE RECREATION.

FOR THE SAFETY AND ENJOYMENT OF EVERYONE THE FOLLOWING TYPES OF BEHAVIOR ARE PROHIBITED IN ZUCCOTTI PARK.

CAMPING AND/OR THE ERECTION OF TENTS OR OTHER STRUCTURES.

LYING DOWN ON THE GROUND, OR LYING DOWN ON BENCHES, SITTING AREAS OR WALKWAYS WHICH UNREASONABLY INTERFERES WITH THE USE OF BENCHES, SITTING AREAS OR WALKWAYS BY OTHERS.

THE PLACEMENT OF TARPS OR SLEEPING BAGS OR ANY OTHER COVERING ON THE PROPERTY.

STORAGE OR PLACEMENT OF PERSONAL PROPERTY ON THE GROUND, BENCHES, SITTING AREAS OR WALKWAYS WHICH UNREASONABLY INTERFERES WITH THE USE OF SUCH AREAS BY OTHERS.

THE USE OF BICYCLES, SKATEBOARDS AND ROLLER BLADES.

REMOVAL OF OBJECTS FROM TRASH RECEPTACLES.

The last line is ironic: the protesters are keeping the park clean precisely by breaking the rule about removal of objects from trash receptacles. The protesters are removing all trash when they get full, and replacing the full trash bag with an empty one, and placing the bag on the curb so that it can be picked up by the city.

---

This was in response to my e-mail request:

Hi! I was hoping you would be nice enough to have one of your people answer a quick question?

Why do you guys have a park? I would assume you have to pay taxes on it, but receive no financial benefit. Is there a rule that you have to set aside a certain amount of space for other buildings you own nearby?

Also, do you have an official statement on the protest? It seems you've been very nice, but on the other hand, it's hard to imagine that you'd be pleased with the protest if it lasts for months.

Thanks you for any response.

October is Cybersecurity Awareness Month -- or is it?
Mon, 03 Oct 2011 17:05:00 +0000

Last year, the president declared October to be "Cybersecurity Awareness Month". But, October has already been Breast Cancer Awareness Month for the pat 25 years.

So which is it? Cybersecurity or Breast Cancer?

The easy answer would be "both", but that's silly. Why not, then, make it "everything awareness month"? Indeed, why don't we make every month Everything Awareness Month.

Choosing both would teach a bad lesson. Everything we do to make cyberspace more secure comes with tradeoffs making cyberspace less useful. If we measured cybersecurity only by what is most secure, then we'd turn of the computer, cut the wires, and bury it. That'll keep the hackers out.

Thus, cybersecurity is about choosing between tradeoffs. It recognizes that we can't endless ask for our budget to increase, but must work within the budget we are given. If that means forgoing anti-virus because we spent this year's money on a firewall, then so be it.

That means with an "awareness month", we only have a fixed "awareness" budget. Every dollar spent promoting Cybersecurity awareness means one dollar taken away from Breast Cancer awareness.

It's not just dollars, but attention span budget. Let's say you disregard my advice and increase your budget to promote both. People still have only a limited attention span, and thus, will pay half the attention to both campaigns.

1 in 8 women will get breast cancer in their lifetimes. I'm not aware of anybody dying to a cybersecurity fail. That makes me think breast cancer is a bit more important than cybersecurity.

Protesters wanted to get arrested on Brooklyn Bridge
Sun, 02 Oct 2011 00:47:00 +0000

(For my complete report on the protest, click here.)


In the #OccupyWallStreet protests, there are claims that the police deliberately led protesters onto the roadway, and then arrested them for being on the road blocking traffic.

I don't know, I wasn't there.

But I was at a Starbucks near Zuccoti park listening to two protesters (young white mails with facial hair and pony tails) about an hour before the march. They were talking about how they were going to march to the Brooklyn bridge, and how it was going to disrupt traffic, and how that was going to lead to arrests. They laughed at this, hoping it would happen, because "that'll finally get us on the news".

I didn't get the impression that they were planning to go out on the roadway and disrupt traffic. My impression, though, was they knew it was going to happen, probably because that's what happens when you march a few thousand protesters up to the bridge. Indeed, as the protesters later marched by the Starbucks on the way to the Brooklyn bridge, I noticed occasional people get out onto the roadway, and cops telling them to get back onto the sidewalk.

This is my impression of the protesters. They aren't necessarily lawless or violent, but they do seem interested in pushing the police to their limits. Even though they mostly follow police directions, not a single one (that I talked to) thinks of the police as being equally on their side. Whereas I see the police being tolerant of minor infractions, the protesters complained how the police was constantly harassing them over miner infractions. Their view is that if you aren't with them toting a placard, then you must out to oppress them.

As you can see in this video, the crowd is happy that they are getting arrested.

My point is: the accusation that "it's the police's fault" that protesters were out on the road way is pretty hard to believe.

---

Update: This NYTimes article describes the incident. It doesn't claim that the police deliberately guided them onto the roadway, but that protesters were confused about where to go.

But, as I indicate above, at least some protesters knew that going onto the roadway would get them arrested.

---

Update: This link is a first hand account of somebody that happened to go for a walk with the protesters and got arrested. Before the march:

As we loitered a young woman handed me a flier that described my legal rights and urged me to write down the number of the National Lawyers Guild on my arm.

"You planning on getting arrested today?" I said.

"You never know," she said.

---

Update: Another eyewitness account

---

Looking at photographs, I find core occupiers from the Central Committee among those arrested for being on the roadway. These are the people who should've know what would happen, even if most other protesters didn't.

I was just threatened by #OccupyWallStreet protesters
Fri, 30 Sep 2011 21:00:00 +0000

(For my complete report on the protest, click here.)


I was just threatened by #OccupyWallStreet protesters. They told me that if I didn’t give up my seat, there were going to break this computer I’m typing on.

I chose a seat in the park that was far from the center of action, but apparently, the center of action moves around. The "media team" wanted to comandeer the table I was sitting out in order to setup a tripod to take pictures of an upcoming speech. Some chick carrying camera (pictured on the right) came up to me and demanded that I leave. When I refused, she told me that they were going to setup a tripod on the table and take pictures there anyway, and that since her focus was going to be on taking pictures, she was almost certainly going to "accidentally" step on my computer.

It wasn’t just her making threats. This fascist chick was accompanied by a thug whose face was visibly twitching in anger, saying in a threatening manner "you had better leave".

The irony of populism is that it’s really the first step of facism. Occupy Wall Street is modeled after the Arab Spring protests that occupied central squares in cities, eventually deposing authoritarian regimes. But those regimes themselves got power by "taking back" their country for the people from the previous despots. Everything these dictators did was justified as being in the name of "the people".

The fascist chick’s comments reflected this. Even after I made it clear that I didn’t support the protest, she insist that I help them anyway because they were serving "everyone’s interest". It’s not true, most "everyone" has made it clear they aren’t interested in the protester’s brand of socialism.

Likewise, she expressed the fact that since they'd been occupying the park for two weeks, it were in control of who could sit where. This is exactly how populists become fascists: principles of freedom important when out of power are lost once they gain power. It's how Castro and Che Guevara visciously suppressed dissent once they gained power in Cuba (the famous picture of Che isn't as a revolutionary in the jungle fighting the man, but was taken short after Che's show trials of political opponents). Now that the #OccupyWallStreet protesters control the park, the won't tolerate those who don't obey their commands.

The reason I didn’t give up my seat is because in general, I respond stubbornly to intimidation. For example, last year the TSA detained me for taking pictures in airport security, something that is perfectly legal, allowed the the TSA’s own rules, and from a larger point of view, necessary for keeping authority accountable to the public.

By the way, there was a third guy. When I said "no" to giving up my seat, he smiled, shook my hand, and said "peace". He was the model of how the protesters should be, with the ability to think privately "I think you are a dick, but I'm going to rise above it, shake your hand, and move on". Just become some of the protesters are fascists doesn’t mean they all are. Also, the others didn't carry through on their threat, they simply set up elsewhere. Finally, most protesters were quite happy to debate their point of view with those who disagreed with them, albeit they are painfully condescending.

---

Update: comments over at Reddit take exception to my use of the word "fascist". Of course, I'm using that term in the casual sense, such as the way you might describe a copy beating up a protester as a "fascist". Likewise, I might casually use "anarchist" to describe a punk throwing stones at a cop, even though I know that "anarchist" has a much richer sense; indeed, as a Libertarian, I might describe myself as an "anarchist" in the sense that I oppose government control, but not in the sense that I would throw rocks at cops.

But since you bring it up, there is a lot of overlap between Occupy Wall Street and fascism, such as "hostile to finance capital, plutocracy, the 'power of money', and internationalist economics" (to quote the Wikipedia page on Fascism). It would be wrong to claim that Occupy Wall Street as actually fascist, but where they overlap, it isn't unjust to point this out.

Thinking on the margin (Economics)
Wed, 21 Sep 2011 19:11:00 +0000

By any rational measure, the Internet is secure enough. It's obviously true. The value of the Internet, with the hackers, is far greater than not having the Internet. Credit card companies, despite all the credit card losses, make a net profit on the Internet.

The problem with the security industry, especially so-called "experts", is that they don't know how to measure "enough security". So they fall back to a default position that no matter how much security you have, it's not enough, you need more. Becoming a security expert is insanely easy: just tell people they don't have enough security. Blame security weakenesses on moral weaknesses, such as laziness, greed, corruption, stupidity, and so on.

But while nobody knows how to measure "enough", it turns out that it's easy. The trick is thinking on the edge, on the margin. You calculate it by whether a marginal increase in security is worth the marginal cost.

Take SSL, for example. Is it secure enough? Well, if you ask the question that way, as an absolute, then you've already lost the battle. But if you instead ask about marginal improvements, it starts to look different. For example, let's say that browser vendors were to announce a new policy such that any CA that gives out a bad certificate for major site (Google, Microsoft, etc.) will be permanently removed from the browser. The question is: is the marginal benefits of this policy worth the marginal costs? We can now have a lively debate about this, with each side bringing up benefits/costs that the other side did not consider. But it's a rational way of debating the problem, rather than debating "is SSL secure enough?".

Or take DNSsec. I love it, it should've been done 10 years ago (from one perspective), but on the other hand, I think it's marginal costs exceeds its marginal benefits. It doesn't solve any of the most common attacks that happen today. I suppose the debate is about what happens in the future. Does it end up being a common point of failure (the way CAs are today), or does it enable new innovation in secure technologies for the future? I suspect a little of both.

Consider the TSA. The most common wrong thing said about them is that they, or one of their techniques, don't stop terrorists. For example, people heavily criticize the taking off the shoes. The correct way to analyze this is on the margin. Is the marginal benefit of forcing passengers to take off their shoes worth the marginal cost?

Here is the thing about terrorism: it's oddly elastic. You'd think that a serious suicide bomber would surgically implant a bomb making it 100% undetectable, and thus, all TSA security is meaningless. In fact, few suicide bombers are that rational. Most are stupid, incompetent, or crazy. Most find it too difficult to ignite a shoe or underwear bomb. Nothing the TSA does can stop the next 9/11 attack by competent suicide bombers, but for everything they do, there is probably some incompetent suicide bomber that is stopped by that procedure. So the question isn't whether these procedures work, they do. The question is whether whether every procedure is worth the cost; I would agree with the assessment that most aren't.

CAPTCHA hell
Tue, 20 Sep 2011 21:09:00 +0000

The better spammers get at solving CAPTCHAs, the harder it becomes for humans to prove that they are, in fact, humans. RECAPTCHA, in particular, has become annoying lately. I often fail the first attempt (or 100% of the attempts if going through TOR, for some reason). Here is a list of CAPTCHAs, see if you can solve them: