cns2010handout 7 :: rainbow tables1 computer and network security matt barrie

CNS2010 handout 7 :: rainbow tables 1

computer and network securitymatt barrie

<[email protected]>


Pre-computation attacks

• A rainbow table is a time-space attack on symmetric cyphers and hash functions.

• The idea comes from an earlier attack using pre-computed hash chains…

• Two Phases– Pre-computation Phase– Online Attack Phase (Cryptanalytic Attack)

Pre-computation Phase• The idea comes from an earlier attack using pre-computed hash chains…

start -> h(start) -> h(h(start)) -> h(h(h(start)))….

• We store the start point, end point and length of the chain

• The chains stops after a fixed number of iterations or a “collision” into an existing end-point (in which case we merge chains)

• If we increase the length of our chains we decrease the size of the table but increase the time taken to iterate over each chain.. (the “time/space tradeoff”)

End-point Start-point

... ...

... ...

... ...

Rainbow Table


Online attack phase

Online Attack Phase• To reverse a given hash Y we simply keep hashing it and comparing to

our list of end points (small) .. when we find a match we lookup the start value and repeatedly hash until we get to our value Y again.. The value before that is our preimage..


Rainbow Tables

• First pioneered by Philippe Oechslin as a fast form of time-memory tradeoff, which he implemented in the Windows password cracker 0phcrack (a play on l0phtcrack).

• A rainbow table is slightly modified form of the precomputation attack. A reduce function is used after each hash.

• The reduce function is an “onto” function that maps a hash to a desired password in the character set– lowercase alphanumeric passwords of 8 characters long– case sensitive passwords of 5-16 characters in length– valid UNIX passwords (96 symbols, 8 characters)


Rainbow Tables

• The reduce function is an “onto” function that maps a hash to a desired password in the character set– reduce(hash(a password)) → next password

• After a chain containing a suitable number of passwords is created, the final password in the chain is hashed, and the final hash and the starting password are stored together in the rainbow table.

• To reverse a hash, look for it in the table. If it isn't found, the following is done to get another hash to try:– hash(reduce(a hash)) → next hash

• We keep going until we find the hash in the table. When we find it, we again lookup the start value for the chain and repeatedly hash until we find out value..


Rainbow tables

http://en.wikipedia.org/wiki/Image:Rainbow_table1.svg


Rainbow Tables

• Rainbow tables use a different reduction function for each "link" in a chain, so that when there is a hash collision in two or more chains the chains will not merge so long as collision doesn't occur at the same position in each chain.

• As well as increasing the probability of a correct crack for a given table size, this use of multiple reduction functions approximately doubles the speed.

• The end result is a table that contains statistically high chance of revealing a password within a short period of time, generally less than a minute. The success probability depends on the parameters used to generate it. These include the character set used, password length, chain length, and table count.


Rainbow Tables

1. We want to reverse the hash “re3xes”

2. We apply reduction function R3 and get “rambo” .. we check the table and don’t find it there

3. We then restart using R2 followed by R3 (and keep doing this with 3, 4, 5 reductions until we succeed).

4. We can see that with two reductions we get “linux23” which is in the table

5. We lookup the start value “password” and then start our search of this chain, comparing the hash at each iteration to our target hash “re3xes”. Once we find it we stop, and we discover the password “culture” that generated that hash value..

http://en.wikipedia.org/wiki/Image:Rainbow_table2.svg


Rainbow Tables

• Rainbow Table for LanManager passwords (windows) config #0Charset [ABCDEFGHIJKLMNOPQRSTUVWXYZ ]Keyspace 8353082582Table size 610MbSuccess probability 0.9990Cracks 5-alpha in a few secondshttp://www.antsight.com/zsl/rainbowcrack/demo_rainbowcrack_cfg0.txt

• Rainbow Table for LanManager passwords (windows) config #1Charset [ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789 ]Keyspace 80603140212 Table size 3 GBSuccess probability 0.9904

http://www.antsight.com/zsl/rainbowcrack/demo_rainbowcrack_cfg0.txt


Rainbow Tables

Demonstration of RainbowCrack 1.2 software to crack 5 alpha only windows password in 30 seconds. Below is the screen output of the cracking process. The pwfile file contain 5 windows password hashes in pwdump format. The *.rt file are precomputed hash table files (rainbow table) we generated before with rtgen.exe utility. The rcrack.exe program find all plaintext in 24 seconds, with additional 6 seconds to load the files from disk. This demonstration run on a PC with one P4 3.0GHz processor and 512MB memory.

=============================================================================================================

D:\rainbowcrack-1.2-win>type pwfile user0:1455:686f63d42397d2e30f97d26b0aca50d3:4170b8c7ef39e916ddffb3a2c61a6c61::: user1:1456:9ebd77263561ef2d0d075e2a1597d7bc:d5ec0d453fade7329c81eaa4fd78fc63::: user2:1457:1a993db3c3a8a908eb5e0c18c96b74de:c514fc8b1803747aa8be2fe087289fc2::: user3:1458:377f3240bafa098d5dca0224fb2b1540:7d9ddf6199d27e12f50f5920f14c4dfd::: user4:1459:25a1d3832aaf6f83caa7a34441025581:2eb5e84791b35ba90480004a28787753:::

D:\rainbowcrack-1.2-win>dir k:\rt0\*.rt

Çý¶¯Æ÷ K ÖÐµÄ¾íÊÇ SATA0

¾íµÄÐòÁÐºÅÊÇ 64BE-26CB

k:\rt0 µÄÄ¿Â¼

2003-09-01 09:59 128,000,000 lm_alpha_0_2100x8000000_all.rt

2003-09-01 09:59 128,000,000 lm_alpha_1_2100x8000000_all.rt

2003-09-01 10:00 128,000,000 lm_alpha_2_2100x8000000_all.rt

2003-09-01 10:00 128,000,000 lm_alpha_3_2100x8000000_all.rt

2003-09-01 10:01 128,000,000 lm_alpha_4_2100x8000000_all.rt

5 ¸öÎÄ¼þ 640,000,000 ×Ö½Ú

0 ¸öÄ¿Â¼ 9,026,281,472 ¿ÉÓÃ×Ö½Ú


Rainbow Tables

D:\rainbowcrack-1.2-win>rcrack k:\rt0\*.rt -f pwfile

lm_alpha_0_2100x8000000_all.rt: 128000000 bytes read, disk access time: 3.55 s verifying the file... searching for 10 hashes...

plaintext of 686f63d42397d2e3 is WHJJGXA

plaintext of 1a993db3c3a8a908 is JLXNYLY

plaintext of 377f3240bafa098d is OFCVPLL

plaintext of 5dca0224fb2b1540 is CGRVFOK

plaintext of 25a1d3832aaf6f83 is DRLMYRX

plaintext of caa7a34441025581 is GWEIVEK

cryptanalysis time: 19.64 s

lm_alpha_1_2100x8000000_all.rt: 128000000 bytes read, disk access time: 3.30 s verifying the file...

searching for 4 hashes...

plaintext of 0f97d26b0aca50d3 is JAQPRMN

plaintext of 9ebd77263561ef2d is BMRBAHY

plaintext of 0d075e2a1597d7bc is RHFKPQT

plaintext of eb5e0c18c96b74de is LSKMQQT

cryptanalysis time: 4.39 s

statistics -------------------------------------------------------

plaintext found: 10 of 10 (100.00%)

total disk access time: 6.84 s

total cryptanalysis time: 24.03 s

total chain walk step: 16450006

total false alarm: 15687

total chain walk step due to false alarm: 13952333


Rainbow Tables

• Rainbow Table for LanManager passwords (windows) config #5Charset [ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789!@#$%^&*()-

_+= ]Keyspace 915358891407 (2^39.7) Table size 24 GB Success probability 0.99909

Demo: crack of following windows password: N73k_a7()TUBoK PrFa$=ptRcb^__ z %G)r*EW&2nk# cjST$=W0U*-5CH (zw= ijV$i*vEX

http://www.antsight.com/zsl/rainbowcrack/demo_rainbowcrack_cfg5.txthttp://www.antsight.com/zsl/rainbowcrack/

demo_rainbowcrack_cfg5.wmv


Rainbow Tables

• Rainbow Table for LanManager passwords (windows) config #6Charset [ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789!@#$%^&*()-

_+=~`[]{}|\:;"'<>,.?/ ]Keyspace 7555858447479 (2^42.8)Table size 64 GB Success probability 0.999

Demo: crack of following windows password: }m-6BRz*Cj=J}G D2@,:H?+e5#: $ Ot\KZ?/a/qr4d^ yc~<{1!Oe}l_j| 5~|3&-K^4S#c3q

Broken in minutes..


Rainbow Tables

• Rainbow Table for MD5 (loweralpha-numeric 1-8)Charset [abcdefghijklmnopqrstuvwxyz0123456789 ]Keyspace 2901713047668 Table size 36 GB Success probability 0.9990410 MD5 hashes broken in 35 minutes..

• Rainbow Table for Microsoft Office– 40-bit encrypted files decrypted in 5 minutes on average– One table for MS Word and one table for MS Excel– Table size is 40 GB– 99.9% accuracy MS Office

• The obvious problem here is the precomputation needs only be done once.. And the attack is extremely scalable.

• Of course, the files are now available on bit torrent.. and rainbow tables crackers are now online on websites..

• Salts are one way to defeat rainbow tables..


FPGA Rainbow Tables Cracker

Starting Point Generator Unit

Enable Pause

First Start PointLast Start Point

(56-bit)

Multiplexer

Select

Key(56-bit)

16 Rounds of DES(48-stage Pipeline)

Start Point(56-bit)

DES Inverse Initial Permutation and

Reduction Function Unit

Ciphertext(64-bit)

Intermediate Point(56-bit)

Resume Key Generator

Permuted Plaintext Register

Plaintext(64-bit)

PermutedPlaintext(64-bit)

End Point(56-bit)

Start Point(56-bit)

First Start PointFirst Start Point

(56-bit)

Initial Permutation

Permuted Plaintext(64-bit)

Reset(Active-Low)

Clock

Precomputer Entity(precomputer_nty)

Precomputation Engine Hardware Controller

Rainbow Table Manager(Database Communication Module)

Precomputation Hardware

End-Point

Precomputation Software

PlaintextStartMask

End Mask

FirstStart Point

LastStart Point

Open-Source SQL Database

Start-Point / End-Point Pair

In 2005, Malcom Sumantri and I decided to implement a rainbow tables cracker in FPGAs..


Hardware Controller for Rainbow End-Point Generator Hardware Controller for Rainbow End-Point GeneratorRainbow Table Manager

(Database Communication Module)

End-Point GeneratorHardware

ProspectiveEnd-Point

ProspectiveColumn for Key

Intermediate Key GeneratorHardware

Online Attack Software Controller

Plaintext CiphertextStart Mask

End Mask

ProspectiveKey

PlaintextStart Mask

LastMask

StartPoint

Open-Source SQL Database

End-Point Start-Point

Enable

Pause

16 Rounds of DES

(48-stage Pipeline)

Preoutput(64-bit)

Reset(Active-Low)

Clock

End Point Generator(oa_endpointgenerator_nty)

Controller

Plaintext(64-bit)

Key(56-bit)

ProspectiveEnd-Point

(56-bit)

Column Number(56-bit)

Plaintext(64-bit)

PermutedPlaintext Register

Plaintext(64-bit)

Ciphertext(64-bit)

Start Mask(56-bit)

End Mask(56-bit)

Step Goal Tool Input to Tool Output of Tool

1 Generate end-points from the chosen plaintext/ciphertext pair.

End-Point Generator (Hardware)

Chosen plaintext, chosen ciphertext, start mark, end-mask

Prospective End-Point, Prospective Column Number

2 Perform table lookup on all end-points generated from Step 1.

Online Attack Software Application

End-Points generated from Step 1.

Start Points that corresponds with matching end-points from Step 1.

3 Generate Key from Starting Points found in Step 2.

Intermediate Key Generator (Hardware)

Start-Point and matching column number (from Steps 1 and 2), start-mask, end-mask.

Candidate Key(s)

4 Test validity of Key Online Attack Software Application

Candidate key(s) from Step 3, chosen plaintext, chosen ciphertext.

Key

FPGA Rainbow Tables Cracker

Online System


Experiment and Results

• Experiment– Cryptanalytic attack on 40-bit DES since the resources to break

DES is out-of-reach for the budget in this thesis.– Use Sensory NetworksTM NodalCoreTM C-1000 PCI Card.

• Xilinx® Virtex-II Pro VP-40 FPGA• Flexible chipset architecture to embed our hardware engines.• PCI interface allows for high-speed communications.

• Results– 40-bit DES Rainbow Table can be generated in less than 4 hours.

Table parameters allows for 85% cryptanalytic success probability.• Fastest known implementation in the literature based on

results.– Online attack of 40-bit DES in 30.8 seconds.

0

0.5

1

1.5

2

2.5

Th

rou

gh

pu

t (M

bits

/se

con

d)

Rainbow TablePrecomputer

(Sumantri)

Distinguished PointsPrecomputer

(Quisquater et al)

Comparison of Precomputation Throughput for 40-bit DES


Data Analysis

• Performance-Cost Analysis– Determine the FPGA chip

that provides the highest performance for the lowest cost.

– Synthesized the hardware designs for various Xilinx FPGAs.

– Spartan 3 S-1500 provides the highest performance-cost relative to other Xilinx® FPGA chips.

• Extrapolate the design of a machine to break DES (56-bit key length)

– Result: DES can be broken with 85% success probability in 72 minutes for an approximate cost of US $1,210.

0

500

1,000

1,500

2,000

2,500

3,000

3,500

0 1,000 2,000 3,000 4,000 5,000 6,000 7,000 8,000 9,000Cost (dollar per FPGA chip)

Perfo

rman

ce (e

nd-p

oint

s ge

nera

ted

per s

econ

d)

Spartan 3

Virtex-II

Virtex-II Pro

Virtex 4


Data Analysis

• FPGAs provides a low cost and effective solution to cryptanalysis.

• Rainbow table attacks provide a faster attack time compared to brute-force, but brute-force uses less resources, that is, memory resources.

– For large key sizes, the rainbow table attack becomes infeasible as memory costs is prohibitive.

Potential Attacker Key Length (bits) Cost (US $)

Clever Outsider 56 353

58 1,413

60 5,650

Knowledgeable Insiders 62 22,600

64 90,400

Funded Organization(Large Corporation, Mafia)

66 361,601

68 1.4 million

Funded Organization(Small Government, Terrorist

Networks)

72 24 million

76 370 million

78 1.5 billion

Funded Organization(Large Government Bodies:

US National Security Agency)

80 6 billion

82 24.7 billion

84 94.8 billion

86 380 billion

88 1.5 trillion

Not feasible 92 242 trillion

cns2010handout 7 :: rainbow tables1 computer and network security matt barrie

Documents

rainbow tables4 rainbow

rainbow tables5 rainbow

rainbow tables6 rainbow

rainbow tables8 rainbow

hash value

hash collision

hash functions

final hash