Differences

This shows you the differences between two versions of the page.

--- pp:intro [2017/02/20 09:36]
pdmatei
+++ pp:intro [2019/02/14 18:19] (current)
pdmatei
@@ Line 1: / Line 1: @@
-====== Introduction to Programming Paradigms ======
+===== Introduction to Programming Paradigms =====
-===== Roadmap =====
+==== In how many different ways can we reverse a sequence of integers? ====
-In this lecture, we will examine a few interesting notions:
+We start the lecture by looking at three different Java programs which reverse a sequence of integers.
-  * programming styles
-  * programming paradigms
-  * programming languages
-and the relation between them. The key insight in this lecture is that each programming style/paradigm introduces **constraints** in how programs should be written. These constraints may serve to:
+The //Cook-book// style:
-  * improve the legibility/usability of the code by other programmers
+<code java>
-  * allow for easy improvements/modifications
+public class Rev {
-  * limit the occurrence of bugs, and easing finding and fixing them
+	public static void main (String[] args) {
-  * allow code portability
+		Integer[] v = new Integer[] {1,2,3,4,5,6,7,8,9};
-  * improve performance
+		int i=0;
+		while (i < v.length/2){
+			int t = v[i];
+			v[i] = v[v.length-1-i];
+			v[v.length-1-i] = t;
+			i++;
+		}
+		show(v);
+	}
+}
+</code>
+//Features://
+  * the reversal is implemented over an array, and performed **in-place** by swapping the first half of the array with the second.
+  * the code is **compact** and **efficient** ($math[n/2] swaps over an array of size $math[n]).
+  * the solution modifies the sequence to a reversed one.
-In the first part of the lecture, we introduce and discuss four programs solving the sorting problem. Each program is written in a different programming language and in a different style. The choice of style for each solution highlights most of the aspects which will be discussed in detail in the programming paradigms lecture.
+//Possible usages://
+  * when development speed is critical
+  * when minimising the number of computation steps is critical
-We then introduce the concept of //programming paradigm// and relate it to programming styles and programming languages.
+The //Undergrad-math-teacher-style// style (short //Undergrad// style):
+<code java>
+interface List {
+	public Integer head ();
+	public List tail ();
+}
-===== Style 1: "Cookbook" (C) =====
+class Cons implements List {
+	Integer val;
+	List next;
+	public Cons (Integer val, List next){
+		this.val = val;
+		this.next = next;
+	}
+	@Override
+	public Integer head() {return val;}
+	@Override
+	public List tail() {return next;}
+}
-<code C>
+class Empty implements List {
-#include <stdio.h>
+	@Override
-#include <stdlib.h>
+	public Integer head() {return -1;}
+	@Override
+	public List tail() {return null;}
+}
-#define SWAP(x,y) t = x; x = y; y = t;
+public class V2 {
-void insertion_sort (int* v, int sz) {
+	private static List rev (List x, List y){
-	int i, j, t;
+		if (x instanceof Empty)
+			return y;
+		return rev(x.tail(), new Cons(x.head(),y));
+	}
-	for (i = 1; i<sz; i++){
+	public static List reverse (List l){
-		j = i;
+		return rev(l,new Empty());
-		while (j>0 && v[j-1] > v[j]){
-			SWAP(v[j-1],v[j])
-			j--;
-		}
 	}
-}
-int main (){
-	int* v = malloc(sizeof(int)*7);
+	public static void main (String[] args){
-	v[0] = 1; v[1] = 9; v[2]=4; v[3]= 15; v[4] = 2; v[5] = 6; v[6] = 0;
+		List v = new Cons(1, new Cons(2, new Cons(3, new Empty())));
+		List r = reverse(v);
-	insertion_sort(v,7);
+	}
 }
 </code>
+//Features://
+  * This solution first attempts to **separate** the sequence representation from the **reversal algorithm**.
+  * The sequence is **represented** as a **list** in the ADT-style.
+  * The in the reversal, the internal list representation is abstracted by using constructors (''Empty'' and ''Cons'') together with observers (''head'' and ''tail'').
+  * The code focuses more on **input-output**: reversal takes a list (instead of an array), and produces another list. The strategy relies on an auxiliary function ''rev'' which uses an accumulator to reverse the list. Both ''rev'' and the display function are recursive. However, ''rev'' is **tail-recursive** hence efficient as long as the programming language supports tail-end optimisation (which Java 8 does not support).
-==== Characteristics ====
+//Possible usages://
-  * **procedural**: the program uses //procedures// (e.g. ''insertion_sort'') or macro-definitions (''SWAP''). While these may be seem like //functions//, they behave rather more like //reusable cookbooks// than functions:
+  * When it is necessary to separate the (reversal) algorithm from the sequence representation (e.g. when we want a single reversal algorithm for several types of lists)
-    * ''SWAP(x,y)'' is textually replaced by ''t = x; x = y; y = t;''
+  * After calling ''reverse'' we have two list objects, the original and reversed list. This could be useful to test if the sequence is a palindrome. This programming style is called //purely-functional//: objects are never modified. Instead, new objects are created from existing ones.
-    * ''insertion_sort'' does not return a sorted vector, but simply modifies the array in-place
-  * **stateful**: the actual contents of the array is changed after ''insertion_sort'' has been called. There are (at least) //two states// for the array: before being sorted, and after. The programmer needs to take this into account, since sorting ''//erases//'' the initial ordering in the array.
+The //Industry// style:
-  * **type/representation-dependent**: the sorting procedure relies on the data representation: an array. For instance, sorting a linked-list would require rewriting the procedure from scratch.
+<code java>
-  * **fast**: is written in C, and does not use recursive function calls, with may be costly.
+import java.util.Iterator;
-===== Style 2: "Pipeline" (Haskell) =====
+class RevView<T> implements Iterable<T> {
+	private T[] array;
-<code haskell>
+	public RevView(T[] array){
-type List = [Integer]
+		this.array = array;
+	}
-insertion_sort :: List -> List
+	@Override
-insertion_sort [] = []
+	public Iterator<T> iterator() {
-insertion_sort (x:xs) = insert x (insertion_sort xs)
+		return new Iterator<T>(){
+			private int crtIndex = array.length - 1;
-insert :: Integer -> List -> List
+			@Override
-insert x [] = [x]
+			public boolean hasNext(){
-insert x (y:ys) = if x > y then y:(insert x ys) else x:y:ys
+				return crtIndex >= 0;
-</code>
+			}
-==== Characteristics ====
+			@Override
+			public T next(){
+				return array[crtIndex--];
+			}
-  * **functional**: the implementation relies on two functions ''insertion_sort'' and ''insert'', which are very similar to functions in the mathematical sense: there receive values as parameter and return values.
+			@Override
-  * **stateless**: there is no concept of //state// or //state-change//: the effect of each function call is **only** to return a value.
+			public void remove () {}
-  * **modular**: the implementation is separated in two components: ''insert'' is independent from ''insertion_sort''
+		};
-  * **pipeline**: the logic of ''insertion_sort'' is to recursively-solve a problem of smaller size, and then use ''insert'' to assemble the solution.
+	}
-  * **slow**: using recursive functions may be costly.
+}
-  * **type/representation-dependent** (similar to Style 1)
+public class OORev {
-===== Style 3: "Game of life" (Prolog) ======
+	public static void main (String[] args) {
+		String[] s = new String[]{"1", "2", "3", "4", "5", "6"};
-<code prolog>
-swap([X,Y|Rest], [Y,X|Rest]) :- X > Y.
+		Iterator<String> r = (new RevView<String>(s)).iterator();
-swap([Z|Rest], [Z|Restp]) :- swap(Rest,Restp).
+		while (r.hasNext()){
+			System.out.println(r.next());
-bubblesort(L,R) :- swap(L,Lp), !, bubblesort(Lp,R).
+		}
-bubblesort(L,L).
+	}
+}
 </code>
+//Features://
+  * The solution here focuses on **views**. The sequence is represented as an **array of strings**, but it could be basically any kind of array. The object ''RevView'' is a **view** over the array:
+    * Note that ''RevView'' only holds a reference to the array, not the array itself. This means that the array could be modified from //outside// the ''RevView'' class;
+    * ''RevView'' returns an iterator over the array, which allows **traversing** the array in reverse order;
+  * The solution also separates the sequence representation (and the separation could be improved), and focuses on **viewing** or **traversing** the list rather than constructing another list from the original one;
-==== Characteristics ====
+//Possible usages://
-  * **backtracking**: ''bubblesort'' attempts to swap two values, and if this is possible, it continues execution. In this particular example, we see a //controlled backtracking//, via the operator ''!''(cut), which prevents exploring the entire execution tree. More details about ''!''(cut) will follow.
+  * Such a solution could be used when different **threads** may want to **read** the same sequence - **views** allow inspecting the sequence without modifying it (in our example).
-  * **stateless & stateful**: during the execution of the program, goal satisfaction can trigger variables to be bound (instantiated) to values (which can be interpreted as a side-effect), however once a variable is instantiated it cannot be modified.
+  * Also, it could be used to check properties of the sequence (e.g. palindrome) without duplicating data, using different views over the same sequence.
-  * **slow**: uses guided backtracking which resembles recursive function call.
-  * **declarative**: it is very close to the bubblesort description: //bubblesort swaps two variables ''X,Y'' such that ''X<Y'', as long as such two variables exist//.
-  * **representation/dependent**
-===== Style 4: "Abstractionist" (Scala) =====
+The //Uni-math-teacher style// (short. //Uni// style):
-<code scala>
+<code java>
-trait Sortable[T] {
+interface Op <A,B> {
-	def sortWith (comp:(T, T) => Boolean) : Sortable[T] // comparator
+	public B call (A a, B b);
 }
-trait LinkedList [T] extends Sortable[T] {
+interface Foldable <A,B> {
-	override def sortWith (comp:(T, T) => Boolean) : LinkedList[T]
+	public B fold (Op<A,B> op, B init);
 }
-case class Void[T]() extends LinkedList[T]{
+class List implements Foldable<Integer,List>{
-	override def sortWith (comp : (T, T) => Boolean) : LinkedList[T] = this
-}
+	Integer val;
+	List next;
+	public List (Integer val, List next){
+		this.val = val;
+		this.next = next;
+	}
-case class Cons[T] (h: T, t:LinkedList[T]) extends LinkedList[T]{
+	public List fold (Op<Integer,List> op, List init){
-	override def sortWith (comp : (T, T) => Boolean) : LinkedList[T] = {
+		if (this.next == null)
-		def ins (x:T, rest:LinkedList[T]) : LinkedList[T] = rest match {
+			return op.call(this.val,init);
-			case Void() => Cons(x,Void[T])
+		return this.next.fold(op,op.call(this.val, init));
-			case Cons(h,t) => if (comp(x,h)) Cons(x,rest) else Cons(h,ins(x,t))
-		}
-		ins(h,t.sortWith(comp))
 	}
 }
+public class V4 {
-object Sort {
+	public static void main (String[] args){
-  def main(args: Array[String]): Unit = {
+		List v = new List(1, new List(2, new List(3, null)));
-  	val l = Cons(1,Cons(9,Cons(4,Cons(15,Cons(2,Void())))))
+		List r = v.fold(new Op<Integer,List>(){
-  	println(l.sortWith(_ > _))
+				@Override
-  }
+				public List call (Integer i, List l){
+					return new List(i,l);
+				}
+		},null);
+	}
 }
 </code>
-==== Characteristics ====
+//Features://
+  * The solution relies on the observation that **reversing a list** is a particular type of **folding operation**. Consider the sequence ''1,2,3'', the ''+'' operation and the initial value ''0''. Folding the sequence with ''+'' and ''0'', amounts to summing up the numbers from the sequence:
+<code>
+fold({1,2,3},+,0) =
+ fold({2,3},+,1+0) =
+   fold({3},+,2+1+0) =
+    fold({},+,3+2+1+0) =
++2+1+0
+</code>
+  *  However, the folding operation need not be arithmetic (nor the initial value - a number). Suppose we replace ''+'' by ''cons'' ('':'') and ''0'' by the empty list ''[]''. The result of the folding operation is ''3:2:1:[]'', which is precisely the reversed list. In this solution, the ''fold'' operation is implemented in an abstract fashion, with respect to a generic binary operation ''op'' and a generic initial value ''init''.
+  * List reversal is a **particular case of a fold operation**
+//Possible usages://
+  * This solution is also characteristic to the functional programming style, more specifically, programming with **higher-order functions**. A higher-order function (like ''fold'') takes other functions (like ''op'') and implements some functionality in terms of it. In a functional programming language, implementing (and using) a ''fold'' is much simpler and elegant than in an imperative language (like Java).
+  * Using higher-order functions is very useful when writing programs over large data (for processing like that done in Machine Learning). Such processing is done using a combination of //map-reduce// functions: //map// transforms data uniformly, and //reduce// computes a new (reduced) value from it. Map-reduce programs are easily to parallelise (however, reversal is not a demonstration for that).
+==== Why so many reversals? ====
+Note that **reversal** is an algorithmically trivial task: take the sequence of elements of the collection at hand, be it array of list (or anything else), in their **reverse** order.
+Our point is that, apart from mastering algorithms, a skilled programmer needs **solid knowledge** on programming concepts and on the way programming languages (and the hardware they employ) are designed. In some cases, these concepts may be subtle (e.g. programming with Monads in Haskell) and may supersede the algorithm at hand in complexity. However, they are crucial in developing a **efficient**, **secure** and **correct** applications.
-  * **type-independence**: list sorting is implemented **independently of the type** of elements contained in the list. In order to sort a list, it is sufficient to have a comparison function, here denoted as: ''comp : (T, T) => Boolean''. Type independence is technically realised here via:
+This lecture will focus on different ways of **writing code** for specific algorithms, in different programming languages, with an emphasis on Functional Languages (Haskell) and Logic-based Languages (Prolog).
-    * **parametric polymorphism** or **genericity**: the ability to define entities (e.g. classes) parameterised with respect to a type T which is established at runtime;
-    * **ad-hoc polymorphism** or **function overriding**: the actual implementation of ''sortWith'' is established at runtime, depending on the actual type of the object on which it is called (Void or Cons).
-    * there is another form of type independence: ''sortWith'' is designed to work on any sortable entity, not only lists
-  * **functional**: the actual sorting is very similar in implementation to the //pipeline style//
-  * **slow**: due to recursivity
-  * **stateless**
-===== Programming Paradigms =====
+We have clear metrics for choosing algorithms. Do we also have metrics for **code writing**? For instance:
+  * legibility
+  * compactness (number of code lines)
+  * ease of use (can other programmers use the code easily)
+  * extensibility (can other programmers add modifications to the code easily)
+  * good documentation
-We introduce three //de-facto programming styles//, which we call **paradigms**, by stating some key features. What is the difference between a //paradigm// and a //style// (e.g. any of the 4 above ones)? There is no formal definition for //paradigm// or //style//. However:
+is a plausible list. While some of these criteria overlap and are difficult to assess objectively, programmers more often than not agree that some programs are **well-written while others are poor** (w.r.t. some criteria).
-  * //paradigms// are rather related to //programming language design//. More precisely, a programming language will (preponderently) implement or adhere to a //paradigm//. However, nowadays, this no longer happens (Scala is an example).
-  * //styles// are more specific to programmers, and may blend several aspects of a paradigm.
-Programming paradigms:
+==== How many other ways? ====
-  * **imperative**: programs specify instructions which iteratively change state. It is very close/similar to the machine architecture. It is generally otherwise unconstrained.
-  * **object-oriented**: programs consist of object definitions (classes). Programming is focused on relations between objects, and their traits (methods they implement). Apart from this enforced structure, programming is similar to the imperative style.
-  * **functional**: programs consist of functions in the mathematical sense. Functions can only return values, no other modifications are allowed. Programs are stateless, and sometimes this is enforced by the programming language (e.g. Haskell).
-  * **logic programming**: programs consist of rules of the form '' A => B'' (if A is true, then B must be true). The input of a program is a goal (expressed in predicate logic). Program execution attempts to satisfy a goal, e.g. it is a **proof of the goal**. In this sense, a program can be seen as a set of axioms. For this reason, Prolog - an example of a logic programming language is also called a **theorem-prover**.
-  * **rule-based**: very similar in appearance to logic programming, but with different inference styles.
-===== Do styles conform to paradims? =====
+There are many possible variations (and combinations) to our examples, but a few elements do stand out:
+  * the functional style for **representing a list** - very akin to Abstract Datatypes
+  * the functional style for reversal - relying on recursion, relying on folding
+  * the Object Oriented style for **traversing a list** in the third example - which although is tightly linked to Java Collections, can be migrated to any other object-oriented language.
-  * **imperative**: The //cookbook// style is definitely **imperative**, as all its characteristics conform to it;
+Such **elements of style** are may be called **design patterns**, i.e. generic ways of writing code, which can be deployed for different implementations.
-  * **object-oriented**: The //abstractionist// style is object-oriented.
-  * **functional**: The //pipeline// style is functional. Also, the //abstractionist// style is functional. This is obvious in the actual sort implementation.
-  * **logic-programming**: The //game-of-life// style does conform to the paradigm.
-===== Do programming languages conform to paradigms? =====
+Some elements of style may have some common ground, or require certain traits from the programming language. These latter are called programming **paradigms**. For instance, writing programs as recursive functions, using higher-order functions and representing data as ADTs (recall that constructors //are// functions) are both styles of the **functional** paradigm.
-  * **C/C++**: ''C'' is definitely imperative. It is free of constraints, and very flexible. ''C++'' is object-oriented - it adds typing as a constraint (this statement will be discussed in more detail in the following lectures), but retains flexibility. It is possible for programmers to use a functional style in C/C++ albeit with some limitations, especially in the form of the resulting code.
+In this lecture, we shall go over the following paradigms:
-  * **Java**: is definitely object-oriented and still preserves traces of ''imperativism'', especially with regards to primitive types (which are not objects). As with **C/C++**, a functional style is possible, but, more than C\C++ may seem forced: for instance, we cannot pass functions as parameter, however we can pass objects which implement certain methods (see, for example, the Command design pattern).
+  * imperative
-  * **Scala**: is object-oriented and functional at the same time. It is possible to program in an imperative style in Scala (for instance, it allows for stateful programming like function side-effects). It is also possible to write "strictly-object-oriented" programs (a la Java) as well as functional programs. Most programs will however blend the two.
+  * object oriented
-  * **Scheme (or Lisp)**: is functional, however it is possible to program imperatively in such languages. However such programs are clumsy and rare. It is not possible to write object-oriented-style programs in these languages, as they do not support advanced typing (this will be discussed further on).
+  * **functional**
-  * **Haskell**: is functional and **pure**. This latter term deserves a more elaborate discussion which will follow. In short, in Haskell it is not possible to program in another style but functional. Haskell also supports advanced typing (reliant on Abstract Datatypes) and ad-hoc and parametric polymorphism.
+  * **logical** (or logic programming)
-  * **Prolog**: is the de-facto logic programming language. It supports some restricted form functional programming (function definition). There is no means for defining user types.
+  * **associative** (or rule-based)
-  * **CLIPS**: is a rule-based programming language designed at NASA. Nowadays CLIPS is no longer widely-used, however elements of CLIPS design are incorporated in many domain-specific languages (i.e. languages which are deployed for solving specific kind of problems) from AI. A good example are rule-based languages used for reasoning on ontologies.
+===== Other questions =====
-===== References =====
+  * Is there a **right** programming language? How to people choose programming languages?
+  * Who invented paradigms? (A history between Lisp vs C)
+  * Relationship between paradigms and programs (one-to-one, one-to-many, many-to-many)
+  * How **extensible** can programs be?
-  * [[https://www.crcpress.com/Exercises-in-Programming-Style/Lopes/p/book/9781482227376 | Exercises in programming style ]]