March 17, 2020 © Sam Siewert
CS 332
Programming Language Concepts
Lecture 12 – Alternative Language
Examples
(API Concurrency vs. Built-in
Language Concurrency)
Tell your Story about Programming
Use the oldest methods to make convincing
science and technical arguments …
© Sam Siewert
2
Common PL vs. Alternative PL StoryPick and adventure - Very simple algorithm or app of interest
Research PL that might give advantage over common
Implement and test algorithm/app in both PLs
– 3+ Objective Metrics - measurable and quantifiableSpeed of execution
Size (compiled image, SLOC, assembly statements)
Complexity (e.g. McCabe cyclomatic)
Memory used for common algorithm (global, heap, stack)
CPU loading for common algorithm
Speed of compilation or interpretation
– 3+ Subjective Observations - Majority would agree when viewed side by sideTwo blocks, functions, modules compared - readability
Syntax equivalents
Semantic equivalents
Testimonials by experts
Number of people using, size of largest project, number of projects
Ease of debugging
Ease of learning
Built-in features vs. need to use libraries (paraphrase extension)
Moral of the Story - Stick with Common, or Use Alternative
© Sam Siewert 3
Final Presentation – the ObjectiveDefine terminology and concepts, #1
Use references (expert voice) to support hypothesis, #2, #3
Present data (metrics) and how you gathered, #4
Provide examples (code side by side) and summary of results, #5, your conclusions, #6
Tell a story using Scientific method1. Play with and learn primary and alternate PLs to make
observations
2. Choose a problem to solve and research
3. Form a hypothesis about which PL will help you solve the problem (and related problems) best
4. Compose and complete tests to verify / refute your hypothesis
At least 3 objective metrics
At least 3 qualitative (subjective)
5. Analyze and summarize you results
6. Present final conclusions (positive, negative, mixed outcome)
© Sam Siewert 4
Definition Testimonial
Example Data
Key terms
Concepts,
Theory
Expert
References
Tell a
Story!
Relate
Data
(graphic)
Telling a Research Story
Ask a Question
Do research (what
other experts think)
Hypothesis
Test with experiment
– Objective measures
– Subjective
– Arguments or Proofs
(inductive, deductive,
transductive)
Analyze / Summarize
Conclusion -
Positive/Negative
Outcome
© Sam Siewert 5
Example – Part 1
Does Lisp/Prolog/R/Scheme offer an advantage for
problems I want to solve?
– What problem do I want to solve?
– What algorithm, application, technology interests me?
– What alternate (new) programming language interests me?
What do experts say regarding an alternate
programming language? Problem? Algorithm?
– Start easy – Rosetta Code
– Explore deeper – google.com/scholar
– Go real deep – Hazy library, ask and expert on Research Gate
My hypothesis is – PL A has advantage over B, we
should switch (port) from B to A
© Sam Siewert 6
Example – Part 2Design tests and experiments
– Objective (by measure)
How big (output code size)
How fast (execution time)
Size of source input (lines of code)
Correctness (by induction, deduction, transduction)
– Subjective (by example)
How easy (to write, read, understand)
Debugging
Support documentation
Number of users and trends of use
Arguments and Proofs
– What does each measure and example mean?
– Why? How do you know?
© Sam Siewert 7
Analyze and Summarize
Big picture from each result – overall
– E.g. Scoring and weighting
– Visualizations
– Summary – experts were right/wrong, I was right/wrong, results
are conclusive (positive, negative), inconclusive (mixed), majority
(positive, negative)
Conclusion
– For problem studied PL A is better than B
– I plan to use PL A
– You should also use PL A for the same types of problem (class
of problem – e.g. sorting)
– Exceptions
– Future work
© Sam Siewert 8
Using Rosetta Code – To ExploreSearch for a Common
problem, task, or algorithm
Compare 2 Programming
Language Contributed
Solutions
© Sam Siewert 9
The Problem
Uses a library, “algorithms”, but common
Uses “sort” function, but common
Dig down one more level – to
sort and algorithms
Rosetta Code for Sort
Choose an algorithm (research category)
– Quick sort
– Insertion sort
– Heap sort
– Bubble sort
– Etc.
E.g. Quick sort of keys (verify contributed solutions)
© Sam Siewert 10
Verification – Objective Measures
1. Correct Implementation?
2. How fast can it run on common test hardware?
3. How big is the generated code?
4. How many lines of code for each total?
5. Can the program scale? [Limits – e.g. how many
records can it sort, divide and conquer, parallel?]
© Sam Siewert 11
Assessment - Subjective
1. Is the correct implementation hard to understand?
a. Side by side comparison
b. Libraries used [how much abstraction provided] – fair
comparison?
2. Does the programming language have a good compiler
or interpreter and IDE?
3. Does the PL and IDE provide good debugging?
4. How many people are using the PL?
5. Does it have good documentation and support?
© Sam Siewert 12
Interesting Google Trends – Subjective
Metrichttps://www.google.com/trends/ - Search Trends
Measure of Interest, Affinity (Like) or Confusion?
© Sam Siewert 13As of 4/3/2017
Concurrency - Terminology
Multiple CPU Cores (Flynn’s Taxonomy)– Multi-Core (MIMD, MPMD)
– Vector Processing (SIMD)
– Micro-Parallelism (SIMT)
– Co-processors (SPMD)
Multiple Software Execution Contexts
Concurrent
Parallel
Multi-programming
Multi-threaded
© Sam Siewert 14
Programming Languages with
Built-in Concurrency
Why not all?
Are libraries and OS better?
© Sam Siewert
15
Major ConceptsConcurrent Processing– Processes, Tasks, Threads
1. For Multi-programming to Overlap I/O Blocking or Leverage Multi-core with Shared Memory – POSIX Threads
2. For Multi-core and Multiple CPU Systems for Services and Parallel Applications (Shared Memory) - OpenMP
3. For HPC Clusters to Scale Processing – MPI, OpenMPI
For Tasking with Shared Memory (Threads) #1 and #2, Alternative is Built-in Language Concurrency
1. Ada95 Tasking, Using Linux GNAT, “sudo apt-get install gnat”
2. Java Threads, DrJava or Eclipse on Linux for Java
3. C++ 2011 – FAQ, New C++11 Design Goals, C++11 Features
© Sam Siewert 16
Java Example
© Sam Siewert 17
public class simpleThreads {
private static int gsum=0;
private static class incrementThread implements Runnable {
public void run() {
for(int i=0; i < 100; i++) {
gsum=gsum+1;
System.out.format("incrementThread: gsum=%d\n", gsum); } } }
private static class decrementThread implements Runnable {
public void run() {
for(int i=0; i < 100; i++) {
gsum=gsum-1;
System.out.format("decrementThread: gsum=%d\n", gsum); } } }
public static void main(String args[]) throws InterruptedException {
System.out.println("Starting increment and decrement threads");
Thread inct = new Thread(new incrementThread());
Thread dect = new Thread(new decrementThread());
inct.start();
dect.start();
System.out.println("Main will loop until it can join both threads");
// Option 1 - just wait indefinitely
inct.join();
dect.join();
System.out.println("TEST COMPLETE\n");}}
Java Output
Fairly Deterministic for
10 iterations each
For 100, More Variation
(Chance for Context
Switching)
Are the Outputs Even in
the Right Order? (Check
with Timestamps)
Are the Outputs Valid?
© Sam Siewert 18
Welcome to DrJava.
> run simpleThreads
Starting increment and decrement threads
Main will loop incrementThread: gsum=1
incrementThread: gsum=1
incrementThread: gsum=2
incrementThread: gsum=3
incrementThread: gsum=4
incrementThread: gsum=5
incrementThread: gsum=6
incrementThread: gsum=7
incrementThread: gsum=8
incrementThread: gsum=9
decrementThread: gsum=0
decrementThread: gsum=8
decrementThread: gsum=7
decrementThread: gsum=6
decrementThread: gsum=5
decrementThread: gsum=4
decrementThread: gsum=3
decrementThread: gsum=2
decrementThread: gsum=1
decrementThread: gsum=0
TEST COMPLETE
>
C Pthreads Example
© Sam Siewert 19
int gsum=0;
void *incThread(void *threadp){
int i; threadParams_t *threadParams = (threadParams_t *)threadp;
for(i=0; i<COUNT; i++) gsum++;}
void *decThread(void *threadp){
int i; threadParams_t *threadParams = (threadParams_t *)threadp;
for(i=0; i<COUNT; i++) gsum--; }
int main (int argc, char *argv[])
{
int rc;
int i=0;
threadParams[i].threadIdx=i;
pthread_create(&threads[i],(void *)0,incThread, (void *)&(threadParams[i]) );
i++;
threadParams[i].threadIdx=i;
pthread_create(&threads[i], (void *)0,decThread, (void *)&(threadParams[i]));
for(i=0; i<2; i++)
pthread_join(threads[i], NULL);
printf("TEST COMPLETE\n");
}
C Output
Which Goes First
Appears to Go to
Completion
Could we Count on a
Specific Sequence
We can Count on 0 at
End?
© Sam Siewert 20
$ ./pthread
Decrement thread idx=1, gsum=-1
Decrement thread idx=1, gsum=-2
Decrement thread idx=1, gsum=-3
Decrement thread idx=1, gsum=-4
Decrement thread idx=1, gsum=-5
Decrement thread idx=1, gsum=-6
Decrement thread idx=1, gsum=-7
Decrement thread idx=1, gsum=-8
Decrement thread idx=1, gsum=-9
Decrement thread idx=1, gsum=-10
Increment thread idx=0, gsum=-9
Increment thread idx=0, gsum=-8
Increment thread idx=0, gsum=-7
Increment thread idx=0, gsum=-6
Increment thread idx=0, gsum=-5
Increment thread idx=0, gsum=-4
Increment thread idx=0, gsum=-3
Increment thread idx=0, gsum=-2
Increment thread idx=0, gsum=-1
Increment thread idx=0, gsum=0
TEST COMPLETE
C Output on Multi-Core
Multi-Core has More
Switching
Less of a Sequence
Shows This Really is
Non-Deterministic
Result is Still 0
© Sam Siewert 21
[ssiewert@everest]$ ./pthread
Increment thread idx=0, gsum=1
Increment thread idx=0, gsum=1
Increment thread idx=0, gsum=2
Increment thread idx=0, gsum=3
Increment thread idx=0, gsum=4
Increment thread idx=0, gsum=5
Decrement thread idx=1, gsum=0
Decrement thread idx=1, gsum=5
Decrement thread idx=1, gsum=4
Decrement thread idx=1, gsum=3
Decrement thread idx=1, gsum=2
Decrement thread idx=1, gsum=1
Decrement thread idx=1, gsum=0
Increment thread idx=0, gsum=6
Increment thread idx=0, gsum=0
Increment thread idx=0, gsum=1
Increment thread idx=0, gsum=2
Decrement thread idx=1, gsum=-1
Decrement thread idx=1, gsum=1
Decrement thread idx=1, gsum=0
TEST COMPLETE
More on ConcurrencyCovered in Operating Systems – Fair threading on one Multi-core Platform
Real-Time Operating Systems– AMP – Asymmetric Multi-core Processing (Unbalanced)
– SMP – Symmetric Multi-core Processing (Balanced)
Parallel and High Performance Computing– NUMA vs. UMA (Uniform, Non-Uniform Memory Access)
– Message Passing vs Shared Memory
CUDA – Thread Grids for Data Transformation
MapReduce - Hadoop
© Sam Siewert 22
Computer Architecture TrendsMulti-Core, Many-Core
– E.g. Intel Xeon Phi MICA, not MICA wearable
Vector Processing Instructions – ARM NEON
– Intel SSE
– POWER Architecture ALTIVEC
Vector Co-Processing– GP-GPU (E.g. NVIDIA Tegra X1, Tesla/Kepler/Fermi)
– AMD ATI StreamProc
HP – “The Machine”, Memristor, Optical, Collapsed Memory Hierarchy
IBM – Power8 Architecture
ARM – SoCs [Altera Cyclone V SoC, Qualcomm, NVIDIA, Apple, Texas Instruments, Marvel, …]
General Commodity Software-Defined Clusters [Google, Facebook, Hadoop, Seamicro (Now Defunct)]
Clear Trend toward Concurrency, Threading, Message Passing, MapReduce
© Sam Siewert 23
Discussion
More Latest PL Revisions Include Concurrency as a
Built-in Language Feature – Good? Bad? Don’t-Care?
Should Concurrency Be an Operating System Feature?
Should it be a PL Run-time Feature?
Should it Be Provided by a Library?
Why is Concurrency a Contemporary Issue in Computer
Engineering, Software Engineering and PL Design?
© Sam Siewert 24