ruby & gcs (qconsp 2014)
Post on 10-May-2015
2.507 Views
Preview:
DESCRIPTION
TRANSCRIPT
Ruby & GCsEntendendo Gerenciamento de Memória
"G1 GC is an incremental parallel compacting GC that provides more predictable pause times compared to CMS GC and Parallel Old GC. By
introducing a parallel, concurrent and multi-phased marking cycle, G1 GC can work with
much larger heaps while providing reasonable worst-case pause times."
Obsoletos Atuais
1.8.6 1.9.3
1.8.7 2.0.0
1.9.2 2.1.1
#include<stdio.h> int main() { int *ptr_one; ! ptr_one = (int *)malloc(sizeof(int)); ! if (ptr_one == 0) { printf("ERROR: Out of memory\n"); return 1; } ! *ptr_one = 25; printf("%d\n", *ptr_one); ! free(ptr_one); ! return 0; }
#include<stdio.h> int main() { int *ptr_one; ! ptr_one = (int *)malloc(sizeof(int)); ! if (ptr_one == 0) { printf("ERROR: Out of memory\n"); return 1; } ! *ptr_one = 25; printf("%d\n", *ptr_one); ! free(ptr_one); ! return 0; }
#include<stdio.h> int main() { int *ptr_one; ! ptr_one = (int *)malloc(sizeof(int)); ! if (ptr_one == 0) { printf("ERROR: Out of memory\n"); return 1; } ! *ptr_one = 25; printf("%d\n", *ptr_one); ! free(ptr_one); ! return 0; }
#include<stdio.h> int main() { int *ptr_one; ! ptr_one = (int *)malloc(sizeof(int)); ! if (ptr_one == 0) { printf("ERROR: Out of memory\n"); return 1; } ! *ptr_one = 25; printf("%d\n", *ptr_one); ! free(ptr_one); ! return 0; }
0000 0000
0000 0000
0000 0000
0001 1001
#include<stdio.h> int main() { int *ptr_one; ! ptr_one = (int *)malloc(sizeof(int)); ! if (ptr_one == 0) { printf("ERROR: Out of memory\n"); return 1; } ! *ptr_one = 25; printf("%d\n", *ptr_one); ! free(ptr_one); ! return 0; }
0000 0000
0000 0000
0000 0000
0001 1001
40 bytes livres
Slot (40 bytes)
Heap (ex 6 slots)
Slot (40 bytes)
Heap (ex 6 slots)
Instância de RVALUE
typedef struct RVALUE { union { struct { VALUE flags; /* always 0 for freed obj */ struct RVALUE *next; } free; struct RBasic basic; struct RObject object; struct RClass klass; struct RFloat flonum; struct RString string; struct RArray array; struct RRegexp regexp; struct RHash hash; struct RData data; struct RTypedData typeddata; struct RStruct rstruct; struct RBignum bignum; struct RFile file; struct RNode node; struct RMatch match; struct RRational rational; struct RComplex complex; struct { struct RBasic basic; VALUE v1; VALUE v2; VALUE v3; } values; } as; #if GC_DEBUG const char *file; VALUE line; #endif } RVALUE;
RVALUE
struct RString { struct RBasic basic; union { struct { long len; char *ptr; union { long capa; VALUE shared; } aux; } heap; char ary[RSTRING_EMBED_LEN_MAX + 1]; } as; };
RSTRING(str)->as.heap.ptr
Slot (40 kb)
Heap (ex 6 slots)
Instância de RVALUE
Dados do RVALUE
Slot (40 kb)
Heap (ex 6 slots)
Instância de RVALUE
Dados do RVALUE
Slot (40 kb)
Heap (ex 6 slots)
Instância de RVALUE
Dados do RVALUE
Mark and Sweep GC
Full Marking and Full Sweep GC
• Slots de mesmo tamanho (não há problema com fragmentação)
• Quanto mais heaps, mais lento
• Fase de marcação conservadora
• pode haver “leaks” (ex. constantes)
• Processo “Stop-the-World"
Master Unicorn ou Passenger
Rails App
NGINX
Rails App
Rails App
fork() fork()
request
150mb 150mb 150mb
Copy on Write (CoW)
só copia o trecho da memória quando for modificada
“Mark Bit Map”
Bitmap Marking GC
Full Bitmap Marking and Full Sweep GC
Master Unicorn ou Passenger
Rails App
NGINX
Rails App
Rails App
fork() fork()
request
150mb 100mb 100mb
Mark Sweep Mark Sweep Mark Sweep
Mark Mark Mark
Lazy Sweep GC
Full Bitmap Marking and Lazy Sweep GC
https://gist.github.com/akitaonrails/10212233
http://u.akita.ws/samsaffron_ruby20
RUBY_GC_MALLOC_LIMIT=30000000 \
RUBY_HEAP_MIN_SLOTS=800000 \
bundle exec rails server
RUBY_GC_MALLOC_LIMIT 8.000.000 30.000.000
RUBY_HEAP_MIN_SLOTS 10.000 800.000
env_gc_malloc_limit_in_mb 7.6mb 26.6mb
total_gc_runs 105 65
heaps_with_used_slots 1.089 1.477
total_heaps_allocated 1.958 1.965
total_heaps_allocated_in_mb 30.6mb 30.7mb
heaps_to_allocate_next_gc 869 0
2.49Mb
71.9Mb
GC Anterior
Weak Generational Hypothesis
“Most young objects die young"
Young Generation (Eden)
Old Generation (Tomb)
Generational GC
Copying Garbage Collector
From-Heap
To-Heap
From-Heap
To-Heap
Mark-Compact GC
Mark Copy Compact Heap Swap GC
• “Stop-and-Copy”
• Sweep precisa ir slot a slot, neste é só considerar tudo no primeiro heap como livre
• Gerenciamento interno de ponteiros
• Feito para corrigir fragmentação
• Ruby tem slots de mesmo tamanho
• Não suporta Copy-on-Write
Ruby 2.1.x
Shady vs Non-Shady
Ruby Object
Write Barrier (Non-Shady only)
Ruby Array
Shady Object
Non-Shady Object (Sunny)
Mark bitmap (mark_bits[])
Shady bitmap ( rememberset_bits[])
Objetos "Old" 96.4%
Objetos "Young" 3.6%
Objetos "Shady" 1.5%
Mark Mark Mark
Major GC
Minor GC
2.49Mb
71.9Mb
Restricted Generational GC (RGenGC)
Full M&S for Shady, Generational for Non-Shady
• Partial Markings, Lazy Sweeps = Menos Stop
• Shady objects não vão para o Tomb
• Write Barriers (referência old para new - non-Shady only)
• Compatível com extensões C (Shady)
• Objetos em Tomb só são remarcados em Full Mark
http://u.akita.ws/samsaffron_ruby21
ruby-trunk e ruby 2.1.2 (não lançado)
http://u.akita.ws/samsaffron_ruby21
< 1.3!(+ major GC)
RUBY_GC_MALLOC_LIMIT_MAX=8000000 \
RUBY_GC_OLDMALLOC_LIMIT_MAX=8000000 \
RUBY_GC_HEAP_OLDOBJECT_LIMIT_FACTOR=0.9
bundle exec rails server
força GC por diminuir o teto
basicamente desabilita minor GC
(de volta a Ruby 2.0.0)
RUBY_GC_MALLOC_LIMIT_MAX 3210241024 8000000
RUBY_GC_OLDMALLOC_LIMIT_MAX 12810241024 8000000
RUBY_GC_HEAP_OLDOBJECT_LIMIT_FACTOR 2.0 0.9
RUBY_GC_HEAP_INIT_SLOTS 10000 300000
RUBY_GC_HEAP_FREE_SLOTS 4096 300000
RUBY_GC_HEAP_GROWTH_FACTOR 1.8 1.25
RUBY_GC_HEAP_GROWTH_MAX_SLOTS (no limit) 150000
(obs: monitore, mensure, tuning, mensure de novo!)
http://tmm1.net/ruby21-rgengc/
• Objetos vão para Tomb se sobreviverem 1 GC
• Objetos podem ir para Tomb cedo demais
• Eden - Survivor - Tenured - Permanent
• RGENGC_THREEGEN mode (experimental)
• Parallel Marking (ex. em Thread separada)
• Tri-Color Marking (talvez)
• Futuro: Incremental Major Phase (major GC menores)
Java• Young Generation
• -XX:+UseSerialGC (copying collector)
• -XX:+UseParallelGC (copying collector multi-thread)
• -XX:+UseParNewGC
• Old Generation
• -XX:+UseParallelOldGC
• -XX:+UseConcMarkSweepGC
Master Unicorn ou Passenger
Rails App
NGINX
Rails App
Rails App
request
190ms 190ms 190ms
50ms de Full GC!
Master Unicorn ou Passenger
Rails App
NGINX
Rails App
Rails App
request
140ms 140ms Full GC
Executa Full GC quando não receber request
OOBGC
Out-of-Band Major GC
Pausas foram de 125ms para 50ms
http://tmm1.net/ruby21-oobgc/
http://u.akita.ws/rubymicroscope
Obrigado!@akitaonrails
slideshare.net/akitaonrails www.akitaonrails.com
top related