searching for maximum cliques (implementation details) workshop on clustering and search techniques...
Post on 14-Dec-2015
219 Views
Preview:
TRANSCRIPT
Searching for maximum cliques (implementation details)
WORKSHOP ON CLUSTERING AND SEARCH TECHNIQUES IN LARGE SCALE NETWORKS
This work is partially funded by the Spanish National Government (DPI2010-21247-C02) and CAR (UPM-CSIC)
Pablo San Segundo Carrillo (Associate professor in UPM)
Overview
2
Simple case studies BITSCAN
Brief description of main features Comparison with state of the art
Application to maximum clique in large scale networks (BBMCS) Graph encoding Coloring Computation of a new subproblem (a child subgraph)
Summary
Case study (I): multi-properties
3
enum mode_t{ MODE_A = 0x01, MODE_B = 0x02, MODE_C = 0x04, MODE_D = 0x08}; class Foo{public: void set_modes(int modes){current_modes=modes;} bool clear_modes(int modes){ return current_modes &= ~modes; } bool test_mode(mode_t mode) const{ return current_modes & mode; }private: int current_modes;};
void main(){ Foo myfoo; myfoo.set_modes(MODE_A | MODE_B | MODE_D); myfoo.clear_modes(MODE_D);
if(myfoo.test_mode(MODE_D)) cout<<"MODE_D is active"<<endl;}
enum mode_t{ MODE_A = 0, MODE_B, MODE_C, MODE_D}; class Foo{public: Foo(int nModes):current_modes(nModes){} void set_modes(const bitarray& bb){ current_modes.set_bit(bb); } void clear_modes(const bitarray & bb){
current_modes.erase_bit(bb); } bool test_mode(mode_t mode) const{ return current_modes.is_bit(mode); }private: bitarray current_modes;};
void main(){ Foo myfoo(4); bitarray new_modes(4); new_modes.set_bit(MODE_A); new_modes.set_bit(MODE_C); new_modes.set_bit(MODE_D); myfoo.set_modes(new_modes); bitarray delete_modes(4); delete_modes.set_bit(MODE_D); myfoo.clear_modes(delete_modes);
if(myfoo.test_mode(MODE_D)) cout<<"MODE_D is active"<<endl;}
Case study (II.1): Set representation
4
Membership to a set 1-bit : member 0-bit: not a member
Storage of a subset of natural numbers
Masks (C-C++)
A U B Ab | Bb
A ∩ B Ab & Bb
A – B Ab &~ Bb
(A B)? {Ab &~ Bb } ≠
Case study (II.2): An example
5
#define NUMBER_OF_STUDENTS 100 void main(){ set<int> s_ids; // student ids set<int> e_ids; // exam ids for(int i=1; i<=NUMBER_OF_STUDENTS; i++){ s_ids.insert(i); } //… determine exam ids
//students which did not take the examset<int> s_not_assist;set_difference(s_ids.begin(), s_ids.end(), e_ids.begin(), e_ids.end(),
insert_iterator<set<int>>( s_not_assist, s_not_assist.begin()) );}
#define NUMBER_OF_STUDENTS 100 void main(){ bitarray s_ids(NUMBER_OF_STUDENTS); // student ids bitarray e_ids(NUMBER_OF_STUDENTS); // exam ids s_ids.init_bit(0,NUMBER_OF_STUDENTS-1);
//… determine exam ids
//students which passed the exam bitarray s_not_assist(s_ids); s_not_assist.erase_bit(e_ids);}
O(n/Wsize)
BITSCAN: a C++ library for bitstrings
6
Inspired by optimization requirements found during 10 years of research in combinatorial optimization problems. Implementation of exact algorithms for NP-hard
problems related to graphs (maximum clique, vertex coloring etc.)
Some of these requirements Fast bitscanning loops
Forward and reverse directions Destructive and non-destructive
Sparsity Semi-sparsity
7
Data types Main data types
bitblock bitarray sparse_bitarray watched_bitarra
y
simple_bitarray
simple_sparse_bitarray
bitarray
bbo
watched_bitarray
bitblock
sparse_bitarray
bitblock type: 64bit bit-twiddling
8
#include "pablodev/bitscan/bitscan.h“
void main(){ BITBOARD bb=0xFFF; cout<<"number of 1-bits:"<<bitblock::popc64(bb)<<endl; cout<<“lsb:"<<bitblock::lsb64_intrinsic(bb)<<endl; cout<<“msb:"<<bitblock::msb64_intrinsic(bb)<<endl; //useful masks bitblock::print(bb & bitblock::MASK_0(4,8)); //removes 1-bits in range [4,8] bitblock::print(bb | bitblock::MASK_1(4,8)); //sets 1-bits in range [4,8]}
bitarray type: bitscanning loops
9
#include "pablodev/bitscan/bitscan.h"#define POPULATION_SIZE 100 void main(){ bitarray bba(POPULATION_SIZE); bba.init_bit(0, 9); //first 10 bits to 1 //typical bit scanning loop int nBit=EMPTY_ELEM; bba.init_scan(bbo::NON_DESTRUCTIVE); while(true){ nBit=bba.next_bit(); if(nBit==EMPTY_ELEM) break; cout<<nBit<<" "; //or whatever } bba.print(); }
#include "pablodev/bitscan/bitscan.h"#define POPULATION_SIZE 100 void main(){ bitarray bba(POPULATION_SIZE); bba.init_bit(0, 9); //first 10 bits to 1 //typical bit scanning loop int nBit=EMPTY_ELEM; bba.init_scan(bbo::DESTRUCTIVE); while(true){ nBit=bba.next_bit_del(); if(nBit==EMPTY_ELEM) break; cout<<nBit<<" "; //or whatever } bba.print(); }
sparse_bitarray type
10
S
Sb= 1111111111111111111111111···111Ssb= 0000000000100000000000000···100 class sparse_bitarray: public bbo{
struct elem_t{ int index; BITBOARD bb; };protected: vector<elem_t> m_aBB; int m_MAXBB; //max num. of bitblocks};
Sorted array by index always
Complexity of setting bits?Complexity of deleting bits?Complexity of bit masking with other sparse bitarrays?
sparse_bitarray: example
11
#include "pablodev/bitscan/bitscan.h“#define POPULATION_SIZE 10000000 void main(){ sparse_bitarray bba(POPULATION_SIZE); bba.set_bit(1216, 1230); cout<<"number of bitblocks:"<<bba.number_of_bitblocks()<<endl;
//bit scanning loop int nBit=EMPTY_ELEM; if(bba.init_scan(bbo::NON_DESTRUCTIVE)!=EMPTY_ELEM){ while(true){ nBit=bba.next_bit(); if(nBit==EMPTY_ELEM) break; cout<<nBit<<" "; } }}
watched_bitarray type
12
Conceived for semi-sparse sets Addresses the following issues
Fast empty-set detection Avoids spurious operations between empty bitblocks
1 0 1 0 1 1 0 1 1 0 1 0 1 1 1 1
LC UC
0 0 0 0 1 1 1 1 1 1 1 1 0 0 0 0
Mask
0 0 0 0 1 1 0 1 1 0 1 0 0 0 0 0
0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0
Mask
0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Mask
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
S
Current state of BITSCAN
13
Alpha Lacking proper doc, standardization of names and
namespaces… Easy to use Active development Applied in cutting edge research on efficient algorithms
Source code available in Biicode repository #include “pablodev/bitscan/bitscan.h” https://www.biicode.com/
Tested on Windows and Linux
WE NEED YOUR FEEDBACK!
State of the Art : functionality
14
std::vector<bool> std::bitset boost::dynamic_bitset
find y find_next
NONE OF THEM ADDRESS ANY OF THE PREVIOUS ISSUES
DIRECTLY
State of the Art: efficiency (I)
15
0.000
0.005
0.010
0.015
0.020
0.025
0.030
0.035
0.040
0.045
0.25 0.5 0.75 0.95
Tiem
po (m
s)
Density
Non Destructive Scan
Bitscan
Boost
Bit population 1000
0.000
0.100
0.200
0.300
0.400
0.500
0.600
0.700
0.800
0.900
1.000
0.25 0.5 0.75 0.95
Tiem
po (m
s)
Density
Destructive Scan
Bitscan
Boost
Bit population 20000
0.000
0.001
0.001
0.002
0.002
0.003
0.003
0.004
0.004
0.005
0.005
500 1000 5000 10000 20000
Tiem
po (m
s)
Nº de Bits probados
Population Size
Bitset
Bitscan
Boost
Density of population 0.5
Results over 10.000
repetitions
State of the Art: efficiency (II)
16
BITSCAN is in the core of a number of algorithms for NP-hard problems widely accepted as state of the art by the scientific community: PASS: An exact vertex coloring algorithm BBMC: An exact maximum clique algorithm
These algorithms are known since 2010 and literature does not report superior performance with other state of the art bitstring implementations.
Graph bitarray encoding : GRAPH
18
0
1
4
3
2
Vertices 0 1 2 3 40 x 1 1 0 01 1 x 1 1 02 1 1 x 0 13 0 1 0 x 14 0 0 1 1 x
Adjacency Matrix
0
1
2
bitarray 0bitarray 2bitarray 3bitarray 4
bitarray 1
#include "pablodev/graph/graph.h“#define NUMBER_OF_VERTICES 5 void main(){ //undirected graph ugraph ug(NUMBER_OF_VERTICES); ug.add_edge(0, 1); ug.add_edge(0, 2); ug.add_edge(1, 2); ug.add_edge(1, 3); ug.add_edge(3, 4); //…}
#include "pablodev/graph/graph.h“#define NUMBER_OF_VERTICES 1000000 void main(){ //undirected graph sparse_ugraph ug(NUMBER_OF_VERTICES); ug.add_edge(0, 1); ug.add_edge(0, 2); ug.add_edge(1, 2); ug.add_edge(1, 3); ug.add_edge(3, 4); //…}
Greedy sequential vertex coloring
19
4
31
2
5 C1
C2
C3
1 3
2 4
5
1 2 3
1,3 , 2,4 ,5
C C C C
C
SEQ: GREEDY COLORING PROCEDURE1.Define a vertex ordering2.Color vertices sequentially with the
least possible color
Implementation of SEQ with BITSCAN/GRAPH (I)
20
#include "pablodev/copt/init_color.h“#include "pablodev/graph/graph.h“#define GRAPH_SIZE 5 void main(){
//Ugraph ugraph ug(GRAPH_SIZE); ug.add_edge(0, 1); ug.add_edge(0, 3); ug.add_edge(1, 2); ug.add_edge(1, 4); ug.add_edge(2, 3); ug.add_edge(2, 4); ug.add_edge(3, 4); ug.add_edge(0, 5); InitColor<ugraph> c(ug); bitarray subgraph(GRAPH_SIZE); subgraph.init_bit(0, GRAPH_SIZE-1) cout<<"col_size:"<<c.greedyIndependentSetColoring(subgraph);}
4
31
2
5
Implementation of SEQ with BITSCAN/GRAPH (II)
21
template<>int InitColor<ugraph>::greedyIndependentSetColoring(bitarray & bb){ int pc=bb.popcn64(), col=1, v=EMPTY_ELEM, from=EMPTY_ELEM; bitarray sel(g.number_of_vertices());
bitarray unsel(bb); while(true){ sel=unsel; sel.init_scan(bbo::DESTRUCTIVE); while(true){ if( (v=sel.next_bit_del(from, unsel))==EMPTY_ELEM ) break; //exit condition: all vertices colored if((--pc)==0) return col;
//implicitly computes next vertex in the same color class sel.erase_block(from, g.get_neighbors(v)); } col++; //next color class } return col;}
Computing child subgraph in maximum clique
22
//P is the current subproblem, Pnew the new child
subproblem//v is the current vertex to expand
//Node generation by maskingAND(g.get_neighbors(v), P, Pnew);
inlineBitBoardS& AND (const BitBoardS& lhs, const BitBoardS& rhs, BitBoardS& res){
res.erase_bit(); int i1=0, i2=0;
while(i1!=lhs.m_aBB.size() && i2!=rhs.m_aBB.size() ){ if(lhs.m_aBB[i1].index<rhs.m_aBB[i2].index){
i1++; }else if(rhs.m_aBB[i2].index<lhs.m_aBB[i1].index){
i2++; }else{ //index match
BitBoardS::elem e(lhs.m_aBB[i1].index, lhs.m_aBB[i1].bb & rhs.m_aBB[i2].bb);
res.m_aBB.push_back(e);i1++, i2++;
}
} return res;}
top related