1) O que é? para que serve?

2) Sintax/semantica

3) Ferramentas ( como é que compila, executa )

In our field (Computer Science/Programming), we apply this concept regularly, when we create abstract classes or make functions to hide complex algorithms and provide a cleaner interface.

Sources:

• Concept of Abstraction

• Abstraction in Computer Science

Simplification: It simplifies complex systems, making them more manageable and understandable.

Reusability: Abstraction allows you to reuse code, as you can use abstract concepts to create a blueprint for multiple implementations.

Encapsulation: It helps protect the internal workings of a system, preventing unintended interference.

Maintenance: Abstraction makes it easier to maintain and modify code since changes to the implementation details do not affect the parts of the system that rely on the abstraction.

In its essence, abstraction allows us to make neater, safer and better code.

Sources:

• Abstraction in programming

• Importance of abstraction in system design

Data Abstraction: Defining custom data types to abstract and encapsulate data, such as classes in object-oriented programming.

Function Abstraction: Creating functions or methods that encapsulate a series of operations, allowing them to be called as a single unit.

Module Abstraction: Grouping related functions and data into modules or libraries, hiding internal details.

APIs (Application Programming Interfaces): Providing abstract interfaces for external programs to interact with a system or library without knowing its internal workings (Example: Flask).

Modularity is the concept of breaking down a system or program into smaller, self-contained, and interrelated units called modules. These modules are designed to perform specific functions or tasks and can be developed and maintained independently.

Souces:

• What is abstraction in programming

• Modular approach in programming

Teams are able to work on different modules concurrently, which can be updated or replaced without affecting the whole system, and more can be added as needed.

Sources:

• Importance of modular programming

Functions and Procedures: These are the simplest form of modules, encapsulating a set of operations or tasks.

Classes and Objects: In object-oriented languages, classes encapsulate data and methods, providing a high level of modularity.

Libraries and Packages: Many programming languages support libraries and packages, which contain multiple modules, organized by functionality.

Namespaces: Namespaces help avoid naming conflicts by allowing different modules to have similar names without collision.

Sources:

• What is modular programming

Imperative Programming Languages (C, C++, Java)

Functional Programming Languages (Haskell, Lisp, Erlang)

Object-Oriented Programming Languages (Python, Java, and C#)

Query Languages (SQL)

Assembly Languages (MIPS)

Logic Programming Languages (Prolog, Mercury)

Educational and Beginner-Friendly Languages (Scratch)

Prompt Engineering (ChatGPT) (not recognized as a traditional programming languaged, but was talked about in class and is worth mentioning)

Syntax: The set of rules that dictate how programs written in the language should be structured, including keywords, operators, and punctuation.

Semantics: The meaning of the program, including how statements are executed, data types, and operations.

Alongside them, there are other components, such as:

Variables and Data Types: Languages have mechanisms for declaring and working with variables and different data types (integers, strings, arrays, etc.).

Control Structures: Programming languages provide constructs for controlling the flow of execution, such as conditionals (if, else) and loops (for, while).

Functions and Procedures: The ability to define and use reusable blocks of code.

Input and Output (I/O): Facilities for reading and writing data to and from external sources, like files or the console.

Error Handling: Mechanisms for handling and reporting errors or exceptions in a program.

Orthogonality: A language should have a small set of simple, consistent, and independent constructs that can be combined in various ways.

Efficiency: Languages should allow efficient execution of programs, both in terms of runtime performance and memory usage.

Safety: Languages should help prevent common programming errors, such as null pointer dereferences and buffer overflows.

Abstraction: Languages should provide levels of abstraction, allowing programmers to work at the appropriate level for their task.

Portability: Programs written in a language should be able to run on different platforms with minimal modification.

Interoperability: The ability to work with other languages or systems, such as integrating with libraries written in other languages.

Formal Specifications: Clear and precise definitions of the language's syntax and semantics aid in consistent and error-free implementation.

1. Loading its own bytecode for execution.

2. Storing data values and data structures used during execution.

3. Loading any necessary run-time systems.

Two key regions of memory come into play:

1. Stack: Used for static memory allocation, the stack follows a last-in, first-out (LIFO) structure. It stores execution data for functions as stack frames. Each frame holds data specific to a function and is cleared when the function exits. Stack memory is suitable for finite and static data (known at compile time).

2. Heap: The heap is used for dynamic memory allocation. It allows for more flexible memory usage, but managing it is more complex. Objects allocated on the heap persist until explicitly deallocated.

Sources:

• Wikipedia

• What is memory management

• Memory management in operating system

• Memory management in programming

-Fixed Partitioning memory management: the main memory is divided into several fixed-sized partitions. These partitions can be uniform or varying in size. Each partition can accommodate a single process, and the number of partitions determines how many programs can run simultaneously.

-Dynamic Partitioning memory management: with dynamic partitioning, each process is allocated only the memory it needs when it is loaded for execution. Memory is assigned to processes until physical memory is fully utilized or there isn't enough space left to accommodate new requests.

-Paging: in a paging system, the main memory is divided into equally-sized blocks called frames. These frame sizes match the size of the pages to maximize memory utilization and prevent external fragmentation.

-Segmentation: in segmentation memory management, the main memory is divided into variable-sized segments. Each program is divided into different segments, each corresponding to a specific module or part of the program.

-Virtual Memory: Virtual memory is a technique where the system uses a combination of physical RAM and disk space to provide the illusion of a larger memory space to processes. Pages are swapped in and out of RAM as needed, allowing for more processes to run concurrently.

Souces:

Memory Management in Operating System (OS)

• Performance Overhead: The garbage collector needs to be efficient in order to not slow down the execution of the program. However, it's challenging to design a garbage collector that is both efficient and effective.

• Determining Reachable Objects: Figuring out which objects are still reachable (and hence not garbage) can be complex. Many garbage collectors use a "mark and sweep" approach, where they first mark all objects reachable from the root, and then sweep away everything not marked.

• Memory Fragmentation: Over time, memory can become fragmented, with blocks of free memory interspersed with blocks of used memory. This can make it difficult for the garbage collector to find large contiguous blocks of free memory for new objects.

• Handling Circular References: Objects that reference each other but are not referenced by any other objects can be a challenge to deal with. These objects are technically garbage since they can't be reached from the root, but a simple reference counting garbage collector would not identify them as such.

• Concurrency and Multithreading: In a multithreaded environment, the garbage collector has to coordinate with all threads to ensure that it does not collect an object that is about to be used by a thread or miss an object that has just been abandoned by a thread.

Source:

What are some problems associated with garbage collection?

1. Java Memory Management

Java uses an automatic memory management system called a garbage collector. The memory management in Java divides into two major parts: the method area and the heap area. The method area is a part of the heap memory shared among all threads. It is created when the JVM starts up and is used to store class structures, method data, and constructor field data.

The heap area is where all the objects are stored. The garbage collector is responsible for automatic memory management in this area. It identifies which objects are no longer in use (garbage) and frees up the memory they were occupying.

Java also has four types of references: Strong, Weak, Soft, and Phantom. These references determine the eligibility of objects for garbage collection. Strong references are the most common and are not eligible for garbage collection. Weak, Soft, and Phantom references are used for specific use cases and have different eligibility criteria for garbage collection.

2. Python Memory Management

Python uses a garbage collector for automatic memory management. This means that Python automatically frees up memory that is no longer in use. Python's memory management system consists of several components:

• The PyMem_Malloc function is used to allocate memory.

• The PyMem_Free function is used to deallocate memory.

Python also uses a pymalloc allocator optimized for small objects with a short lifetime. It uses memory mappings called “arenas” with a fixed size.

Python has a tracemalloc module that can provide statistics on allocated memory blocks per filename and per line number, and compute the difference between two snapshots to detect memory leaks.

3. C++ Memory Management

C++ does not have a built-in garbage collector like Python and Java. Therefore, memory management in C++ is largely manual. It involves the use of new and delete keywords to allocate and deallocate memory respectively. The new keyword is used to allocate memory on the heap for an object.

The delete keyword is used to deallocate memory that was previously allocated with new.

However, C++ also provides smart pointers like unique_ptr, shared_ptr, and weak_ptr which can be used to manage memory in a more automatic and safer way.

In conclusion, while Python and Java handle memory management automatically, C++ requires more manual intervention. All three languages provide tools and techniques to manage memory efficiently and prevent memory leaks or other memory-related issues.

Other mentions:

1. C#:

   • Automatic Memory Management: C# uses a garbage collector (GC) to manage memory.

   • Generational GC: Objects start in the young generation and move to the old generation if they survive multiple GC cycles.

   • Dynamic Typing: C# is statically typed but supports dynamic typing through features like reflection.

   • Example: When an object is no longer reachable, the GC frees its memory.

2. Ruby:

   • Automatic Memory Management: Ruby employs a mark-and-sweep garbage collector.

   • Cyclic Garbage Collection: Handles circular references efficiently.

   • Dynamic Typing: Ruby allows variables to change types during execution.

   • Example: Unreachable objects are identified and collected.

Sources:

• Memory management in various languages

• How memory is managed in Python

This approach is used for decreasing the response time of the system by using the single processing unit.

It creates the illusion of parallelism. The chunks of a task aren’t parallelly processed, but inside the application, there is more than one task being processed at a time. It doesn’t fully end one task before it begins ensuing.

Sources:

• Difference between concurrency and parallelism

In other words, it's a concept where multiple operations or tasks are performed at the same time.

This is a key aspect of multithreading and multiprocessing in computer systems, where different tasks can be executed concurrently to improve the overall performance and efficiency of a system.

Sources:

• Difference between concurrency and parallelism

• Introduction to parallel computing

1. Threads: This technique allows a single process to have multiple code segments, like threads. These segments run concurrently within the context of that process.

2. Multiprocessing: The employment of two or more central processing units (CPUs) within a single computer system is known as multiprocessing.

3. Distributed Computing: A distributed computing system consists of multiple computer systems that are run as a single system. The computers in a system can be physically close to each other and connected by a local network, or they can be distant and connected by a wide area network.

4. Libraries and Frameworks: Some languages provide libraries or frameworks that support concurrency and parallelism. For example, the POSIX thread library for C programming language allows implementing almost all of the concurrent programming concepts, such as semaphores, multi-threads, and condition variables.

5. Asynchronous Programming: Asynchronous processing means that your program performs non-blocking operations and can provide some level of concurrency. This is how concurrency is mainly achieved in single-threaded languages like JavaScript.

Sources:

• Concurrency vs Paralellism (Baeldung)

• Concurrency vs Parallelism (Devopedia)

• Concurrency in modern languages

1. Improved Performance: Concurrency and parallelism can significantly improve the performance of applications. By allowing multiple tasks to be executed simultaneously, applications can process more data in less time, leading to faster response times and improved user experience.

2. Efficient Resource Utilization: Concurrency and parallelism can help to utilize CPU, memory, and network resources more efficiently. For example, in a parallel computing environment, different tasks can be delegated to different processors, allowing the system to make better use of its resources.

3. Enhanced Responsiveness: Concurrency and parallelism can make applications more responsive. By allowing an application to start processing one task while waiting for the response from another task, concurrency can help to keep the application responsive and prevent it from becoming unresponsive or freezing.

4. Increased Throughput: Concurrency and parallelism can increase the throughput of an application. This means that the application can handle more tasks or requests in a given amount of time. This is particularly beneficial in scenarios where the application needs to process a large number of tasks or requests, such as in a high-load server environment.

5. Better Utilization of Multi-Core Systems: In a multicore system, concurrency and parallelism can help to make better use of the multiple cores. Each core can execute a different task, allowing the system to process multiple tasks simultaneously and improve the overall performance.

   
   

In conclusion, concurrency and parallelism are powerful tools that can significantly improve the performance, responsiveness, and resource utilization of applications.

However, they also introduce additional complexity and potential for errors, which will be discussed in the next question.

Sources:

• Advantages of Concurrency and Parallelism

1. Complexity and Debugging: Concurrency and parallelism introduce complexity into the code. It can be difficult to design and debug concurrent or parallel systems due to the overlapping and interleaved execution of tasks. This complexity arises from the need to manage multiple tasks or threads, handle synchronization, and deal with issues like race conditions and deadlocks.

2. Data Consistency and Synchronization: Concurrency and parallelism can lead to issues with data consistency and synchronization. For example, if multiple threads or processes are trying to access or modify the same data simultaneously, it can lead to inconsistent states or race conditions. Managing data consistency and synchronization is a significant challenge in concurrent and parallel programming.

3. Resource Management: Concurrency and parallelism can lead to issues with resource management. For example, if too many threads or processes are created, it can exhaust system resources and lead to performance degradation or system crashes. Managing system resources effectively is a key challenge in concurrent and parallel programming.

4. Non-Determinism: Concurrency and parallelism can introduce non-determinism into the program execution. This means that the order in which tasks or threads are scheduled and executed can vary, leading to different outcomes each time the program runs. This non-determinism can make it difficult to predict, test, and debug the program.

5. Error Handling and Recovery: Concurrency and parallelism can make error handling and recovery more complex. If an error occurs in one thread or process, it can potentially affect other threads or processes. This can make it difficult to isolate and recover from errors. Furthermore, if a thread or process fails, it can lead to system-wide failures if not properly handled.

6. Scalability: As the number of tasks or threads increases, the challenges of managing and synchronizing these tasks or threads can become more significant. This can limit the scalability of concurrent and parallel systems.

   
   

Sources:

• Concurrency vs Parallelism (& Issues In Concurrent Systems)

• The Promise and Challenges of Concurrency

1. Understanding Concurrency and Parallelism: Before you start writing concurrent and parallel programs, it's important to understand the concepts of concurrency and parallelism. Concurrency is about dealing with lots of things at once, while parallelism is about doing lots of things at once. They are not the same thing. Concurrency is about structure, while parallelism is about execution.

2. Using Appropriate Tools and Techniques: Depending on the programming language you are using, there are different tools and techniques you can use to write concurrent and parallel programs. For example, in Python, you can use multithreading, asynchronous programming, and multiprocessing capabilities. In Java, you can use threads, locks, shared memory, parallel streams for bulk data, transactional memory, message passing, and lock-free data structures with compare-and-swap.

3. Handling Shared State: When writing concurrent and parallel programs, it's important to handle shared state correctly to avoid race conditions and other concurrency issues. This can be done by using locks, atomic operations, or other synchronization techniques.

4. Testing and Debugging: Testing and debugging concurrent and parallel programs can be challenging due to the non-deterministic nature of concurrent and parallel execution. It's important to use appropriate testing strategies and tools to ensure the correctness of your programs.

5. Performance Measurement and Optimization: Finally, it's important to measure and understand the performance and scalability of your concurrent and parallel programs. This can be done using profiling tools and techniques. Once you understand where the performance bottlenecks are, you can then optimize your programs accordingly.

Sources:

• Techniques for Optimizing Performance

• Pratical Concurrent and Parallel Programming

They allow developers to write code that can run on multiple processors simultaneously, significantly improving performance for tasks that can be broken down into independent subtasks.

Parallel Programming Models

Parallel programming models are abstractions of parallel computer architecture, which provide a convenient way to express algorithms and their composition in programs.

They can be broadly classified into two areas: process interaction and problem decomposition.

1. Process Interaction: This involves the mechanisms by which parallel processes communicate with each other. Common forms of interaction include shared memory and message passing.

• Shared Memory: In a shared-memory model, parallel processes share a global address space that they read and write to asynchronously. This model is supported by many parallel programming languages and libraries, such as Cilk, OpenMP, and Threading Building Blocks.

• Partitioned Global Address Space (PGAS): PGAS models provide a middle ground between shared memory and message passing. They offer a global memory address space abstraction that is logically partitioned, where a portion is local to each process. This model is supported by languages and libraries such as Fortran 2008, Chapel, UPC++, and SHMEM.

  
  

2. Problem Decomposition: This involves breaking down a problem into smaller subtasks that can be solved concurrently. Common forms of problem decomposition include task parallelism, data parallelism, and stream parallelism.

• Task Parallelism: This model focuses on processes or threads of execution, which are often behaviorally distinct, emphasizing the need for communication.

• Data Parallelism: This model focuses on performing operations on a data set, typically a regularly structured array. A set of tasks will operate on this data, but independently on disjoint partitions.

• Stream Parallelism: Also known as pipeline parallelism, this model focuses on dividing a computation into a sequence of stages, where each stage processes a portion of the input data. Each stage operates independently and concurrently.

  
  

Parallel Programming Frameworks

Parallel programming frameworks provide a structured way to write parallel code. They often include libraries and tools that simplify the process of parallelizing code, handling communication between processes, and managing resources.

1. Charm++: Charm++ is a parallel programming framework that uses a unified data/task parallel model. It allows developers to express parallelism in terms of interacting collections of objects and provides asynchronous methods for communication. The framework also includes an adaptive runtime system that optimizes performance.

2. HiCOPS: HiCOPS is a parallel computational framework implemented using object-oriented C++17, MPI, OpenMP, Python, Bash, and CMake. It provides command-line tools for MPI task mapping, database processing, file format conversion, and result post-processing.

3. JAUMIN: JAUMIN is a programming framework for large-scale numerical simulation on unstructured meshes. It provides a high-level interface for defining and running parallel simulations.

   
   

These models and frameworks are crucial in HPC for developing efficient parallel code, enabling the effective utilization of multi-core and multi-processor systems.

Sources:

• Parallel programming model

• Charm++

• Many emerging languages focus on efficient handling of concurrent and parallel programming to leverage multi-core architectures effectively.

Functional Programming:

• A growing interest in functional programming languages that treat computation as the evaluation of mathematical functions without changing state.

Domain-Specific Languages (DSLs):

• There's an emphasis on creating languages tailored to specific problem domains, providing more expressive power for certain types of tasks.

Safety and Memory Management:

• Modern languages often prioritize safety and memory management to reduce the risk of common programming errors like null pointer dereferences or buffer overflows.

Ease of Learning and Productivity:

• Some emerging languages focus on simplicity, readability, and ease of learning to improve developer productivity.

1. Increased Productivity:

   • Higher-level abstractions and better tooling can contribute to increased developer productivity.

2. Concurrency Efficiency:

   • Improved support for concurrent and parallel programming can lead to more efficient use of modern hardware.

3. Reduced Errors:

   • Enhanced safety features can help catch and prevent common programming errors.

Challenges:

1. Learning Curve:

   • Developers may face a learning curve when adopting a new paradigm or language, potentially impacting initial productivity.

2. Compatibility and Integration:

   • Integrating new languages into existing ecosystems and ensuring compatibility can be challenging.

3. Community and Ecosystem:

   • The success of a programming language often depends on the growth of its community and the availability of libraries and tools.

• Rust emphasizes performance, memory safety, and concurrency, making it suitable for systems programming.

Elixir (Functional and Concurrent Programming):

• Elixir, built on the Erlang VM, is designed for concurrent and distributed systems.

Kotlin (Conciseness and Interoperability):

• Kotlin, interoperable with Java, aims for conciseness and expressiveness.

As the landscape diversifies, developers are presented with a richer array of tools, fostering a culture of choice and diversity in problem-solving approaches.

The increasing importance of data security and system reliability drives a heightened focus on languages with advanced safety features, shaping a future where robust code is not merely a preference but a necessity.

Evolving development workflows and best practices are likely as programmers adapt to the unique characteristics of new languages and paradigms.

The trend toward hybrid approaches, wherein multiple languages coexist within a single project to leverage their respective strengths for different components, is gaining prominence.

Overall, these shifts signal a dynamic future where developers navigate an ever-expanding toolbox, necessitating continuous learning and adaptability to remain at the forefront of effective and efficient software development.