CS_learn/hello-algo
1
0
Fork 0
mirror of https://github.com/krahets/hello-algo.git synced 2026-04-29 04:50:56 +08:00
Code Issues Packages Projects Releases Wiki Activity

deploy

Browse Source
This commit is contained in:
krahets
2025-01-14 03:42:26 +08:00
parent 373141a961
commit 0cfb62d8f9
16 changed files with 135 additions and 115 deletions
Download Patch File Download Diff File Expand all files Collapse all files

48
en/chapter_array_and_linkedlist/ram_and_cache/index.html
View File

@@ -3602,10 +3602,10 @@
<!-- Page content -->
<h1 id="44-memory-and-cache">4.4 &nbsp; Memory and cache *<a class="headerlink" href="#44-memory-and-cache" title="Permanent link">&para;</a></h1>
<p>In the first two sections of this chapter, we explored arrays and linked lists, two fundamental and important data structures, representing "continuous storage" and "dispersed storage" respectively.</p>
<p>In fact, <strong>the physical structure largely determines the efficiency of a program's use of memory and cache</strong>, which in turn affects the overall performance of the algorithm.</p>
<p>In the first two sections of this chapter, we explored arrays and linked lists, two fundamental data structures that represent "continuous storage" and "dispersed storage," respectively.</p>
<p>In fact, <strong>the physical structure largely determines how efficiently a program utilizes memory and cache</strong>, which in turn affects the overall performance of the algorithm.</p>
<h2 id="441-computer-storage-devices">4.4.1 &nbsp; Computer storage devices<a class="headerlink" href="#441-computer-storage-devices" title="Permanent link">&para;</a></h2>
<p>There are three types of storage devices in computers: <u>hard disk</u>, <u>random-access memory (RAM)</u>, and <u>cache memory</u>. The following table shows their different roles and performance characteristics in computer systems.</p>
<p>There are three types of storage devices in computers: <u>hard disk</u>, <u>random-access memory (RAM)</u>, and <u>cache memory</u>. The following table shows their respective roles and performance characteristics in computer systems.</p>
<p align="center"> Table 4-2 &nbsp; Computer storage devices </p>
<div class="center-table">
@@ -3645,56 +3645,56 @@
</tr>
<tr>
<td>Price</td>
<td>Cheaper, several cents to yuan / GB</td>
<td>More expensive, tens to hundreds of yuan / GB</td>
<td>Cheaper, a few cents to a few dollars / GB</td>
<td>More expensive, tens to hundreds of dollars / GB</td>
<td>Very expensive, priced with CPU</td>
</tr>
</tbody>
</table>
</div>
<p>We can imagine the computer storage system as a pyramid structure shown in Figure 4-9. The storage devices closer to the top of the pyramid are faster, have smaller capacity, and are more costly. This multi-level design is not accidental, but the result of careful consideration by computer scientists and engineers.</p>
<p>The computer storage system can be visualized as a pyramid, as shown in Figure 4-9. The storage devices at the top of the pyramid are faster, have smaller capacities, and are more expensive. This multi-level design is not accidental, but a deliberate outcome of careful consideration by computer scientists and engineers.</p>
<ul>
<li><strong>Hard disks are difficult to replace with memory</strong>. Firstly, data in memory is lost after power off, making it unsuitable for long-term data storage; secondly, the cost of memory is dozens of times that of hard disks, making it difficult to popularize in the consumer market.</li>
<li><strong>It is difficult for caches to have both large capacity and high speed</strong>. As the capacity of L1, L2, L3 caches gradually increases, their physical size becomes larger, increasing the physical distance from the CPU core, leading to increased data transfer time and higher element access latency. Under current technology, a multi-level cache structure is the best balance between capacity, speed, and cost.</li>
<li><strong>Replacing hard disks with memory is challenging</strong>. Firstly, data in memory is lost after power off, making it unsuitable for long-term data storage; secondly, memory is significantly more expensive than hard disks, limiting its feasibility for widespread use in the consumer market.</li>
<li><strong>Caches face a trade-off between large capacity and high speed</strong>. As the capacity of L1, L2, and L3 caches increases, their physical size grows, increasing the distance from the CPU core. This results in longer data transfer times and higher access latency. With current technology, a multi-level cache structure provides the optimal balance between capacity, speed, and cost.</li>
</ul>
<p><a class="glightbox" href="../ram_and_cache.assets/storage_pyramid.png" data-type="image" data-width="100%" data-height="auto" data-desc-position="bottom"><img alt="Computer storage system" class="animation-figure" src="../ram_and_cache.assets/storage_pyramid.png" /></a></p>
<p align="center"> Figure 4-9 &nbsp; Computer storage system </p>
<div class="admonition tip">
<p class="admonition-title">Tip</p>
<p>The storage hierarchy of computers reflects a delicate balance between speed, capacity, and cost. In fact, this kind of trade-off is common in all industrial fields, requiring us to find the best balance between different advantages and limitations.</p>
<p>The storage hierarchy in computers reflects a careful balance between speed, capacity, and cost. This type of trade-off is common across various industries, where finding the optimal balance between benefits and limitations is essential.</p>
</div>
<p>Overall, <strong>hard disks are used for long-term storage of large amounts of data, memory is used for temporary storage of data being processed during program execution, and cache is used to store frequently accessed data and instructions</strong> to improve program execution efficiency. Together, they ensure the efficient operation of computer systems.</p>
<p>As shown in Figure 4-10, during program execution, data is read from the hard disk into memory for CPU computation. The cache can be considered a part of the CPU, <strong>smartly loading data from memory</strong> to provide fast data access to the CPU, significantly enhancing program execution efficiency and reducing reliance on slower memory.</p>
<p>Overall, <strong>hard disks provide long-term storage for large volumes of data, memory serves as temporary storage for data being processed during program execution, and cache stores frequently accessed data and instructions to enhance execution efficiency</strong>. Together, they ensure the efficient operation of computer systems.</p>
<p>As shown in Figure 4-10, during program execution, data is read from the hard disk into memory for CPU computation. The cache, acting as an extension of the CPU, <strong>intelligently preloads data from memory</strong>, enabling faster data access for the CPU. This greatly improves program execution efficiency while reducing reliance on slower memory.</p>
<p><a class="glightbox" href="../ram_and_cache.assets/computer_storage_devices.png" data-type="image" data-width="100%" data-height="auto" data-desc-position="bottom"><img alt="Data flow between hard disk, memory, and cache" class="animation-figure" src="../ram_and_cache.assets/computer_storage_devices.png" /></a></p>
<p align="center"> Figure 4-10 &nbsp; Data flow between hard disk, memory, and cache </p>
<h2 id="442-memory-efficiency-of-data-structures">4.4.2 &nbsp; Memory efficiency of data structures<a class="headerlink" href="#442-memory-efficiency-of-data-structures" title="Permanent link">&para;</a></h2>
<p>In terms of memory space utilization, arrays and linked lists have their advantages and limitations.</p>
<p>On one hand, <strong>memory is limited and cannot be shared by multiple programs</strong>, so we hope that data structures can use space as efficiently as possible. The elements of an array are tightly packed without extra space for storing references (pointers) between linked list nodes, making them more space-efficient. However, arrays require allocating sufficient continuous memory space at once, which may lead to memory waste, and array expansion also requires additional time and space costs. In contrast, linked lists allocate and reclaim memory dynamically on a per-node basis, providing greater flexibility.</p>
<p>On the other hand, during program execution, <strong>as memory is repeatedly allocated and released, the degree of fragmentation of free memory becomes higher</strong>, leading to reduced memory utilization efficiency. Arrays, due to their continuous storage method, are relatively less likely to cause memory fragmentation. In contrast, the elements of a linked list are dispersedly stored, and frequent insertion and deletion operations make memory fragmentation more likely.</p>
<p>On one hand, <strong>memory is limited and cannot be shared by multiple programs</strong>, so optimizing space usage in data structures is crucial. Arrays are space-efficient because their elements are tightly packed, without requiring extra memory for references (pointers) as in linked lists. However, arrays require pre-allocating a contiguous block of memory, which can lead to waste if the allocated space exceeds the actual need. Expanding an array also incurs additional time and space overhead. In contrast, linked lists allocate and free memory dynamically for each node, offering greater flexibility at the cost of additional memory for pointers.</p>
<p>On the other hand, during program execution, <strong>repeated memory allocation and deallocation increase memory fragmentation</strong>, reducing memory utilization efficiency. Arrays, due to their continuous storage method, are relatively less likely to cause memory fragmentation. In contrast, linked lists store elements in non-contiguous locations, and frequent insertions and deletions can exacerbate memory fragmentation.</p>
<h2 id="443-cache-efficiency-of-data-structures">4.4.3 &nbsp; Cache efficiency of data structures<a class="headerlink" href="#443-cache-efficiency-of-data-structures" title="Permanent link">&para;</a></h2>
<p>Although caches are much smaller in space capacity than memory, they are much faster and play a crucial role in program execution speed. Since the cache's capacity is limited and can only store a small part of frequently accessed data, when the CPU tries to access data not in the cache, a <u>cache miss</u> occurs, forcing the CPU to load the needed data from slower memory.</p>
<p>Although caches are much smaller in space capacity than memory, they are much faster and play a crucial role in program execution speed. Due to their limited capacity, caches can only store a subset of frequently accessed data. When the CPU attempts to access data not present in the cache, a <u>cache miss</u> occurs, requiring the CPU to retrieve the needed data from slower memory, which can impact performance.</p>
<p>Clearly, <strong>the fewer the cache misses, the higher the CPU's data read-write efficiency</strong>, and the better the program performance. The proportion of successful data retrieval from the cache by the CPU is called the <u>cache hit rate</u>, a metric often used to measure cache efficiency.</p>
<p>To achieve higher efficiency, caches adopt the following data loading mechanisms.</p>
<ul>
<li><strong>Cache lines</strong>: Caches don't store and load data byte by byte but in units of cache lines. Compared to byte-by-byte transfer, the transmission of cache lines is more efficient.</li>
<li><strong>Prefetch mechanism</strong>: Processors try to predict data access patterns (such as sequential access, fixed stride jumping access, etc.) and load data into the cache according to specific patterns to improve the hit rate.</li>
<li><strong>Spatial locality</strong>: If data is accessed, data nearby is likely to be accessed in the near future. Therefore, when loading certain data, the cache also loads nearby data to improve the hit rate.</li>
<li><strong>Cache lines</strong>: Caches operate by storing and loading data in units called cache lines, rather than individual bytes. This approach improves efficiency by transferring larger blocks of data at once.</li>
<li><strong>Prefetch mechanism</strong>: Processors predict data access patterns (e.g., sequential or fixed-stride access) and preload data into the cache based on these patterns to increase the cache hit rate.</li>
<li><strong>Spatial locality</strong>: When a specific piece of data is accessed, nearby data is likely to be accessed soon. To leverage this, caches load adjacent data along with the requested data, improving hit rates.</li>
<li><strong>Temporal locality</strong>: If data is accessed, it's likely to be accessed again in the near future. Caches use this principle to retain recently accessed data to improve the hit rate.</li>
</ul>
<p>In fact, <strong>arrays and linked lists have different cache utilization efficiencies</strong>, mainly reflected in the following aspects.</p>
<p>In fact, <strong>arrays and linked lists have different cache utilization efficiencies</strong>, which is mainly reflected in the following aspects.</p>
<ul>
<li><strong>Occupied space</strong>: Linked list elements occupy more space than array elements, resulting in less effective data volume in the cache.</li>
<li><strong>Cache lines</strong>: Linked list data is scattered throughout memory, and since caches load "by line," the proportion of loading invalid data is higher.</li>
<li><strong>Prefetch mechanism</strong>: The data access pattern of arrays is more "predictable" than that of linked lists, meaning the system is more likely to guess which data will be loaded next.</li>
<li><strong>Spatial locality</strong>: Arrays are stored in concentrated memory spaces, so the data near the loaded data is more likely to be accessed next.</li>
<li><strong>Occupied space</strong>: Linked list elements take up more space than array elements, resulting in less effective data being held in the cache.</li>
<li><strong>Cache lines</strong>: Linked list data is scattered throughout the memory, and cache is "loaded by row", so the proportion of invalid data loaded is higher.</li>
<li><strong>Prefetch mechanism</strong>: The data access pattern of arrays is more "predictable" than that of linked lists, that is, it is easier for the system to guess the data that is about to be loaded.</li>
<li><strong>Spatial locality</strong>: Arrays are stored in a continuous memory space, so data near the data being loaded is more likely to be accessed soon.</li>
</ul>
<p>Overall, <strong>arrays have a higher cache hit rate and are generally more efficient in operation than linked lists</strong>. This makes data structures based on arrays more popular in solving algorithmic problems.</p>
<p>It should be noted that <strong>high cache efficiency does not mean that arrays are always better than linked lists</strong>. Which data structure to choose in actual applications should be based on specific requirements. For example, both arrays and linked lists can implement the "stack" data structure (which will be detailed in the next chapter), but they are suitable for different scenarios.</p>
<p>It should be noted that <strong>high cache efficiency does not mean that arrays are always better than linked lists</strong>. The choice of data structure should depend on specific application requirements. For example, both arrays and linked lists can implement the "stack" data structure (which will be detailed in the next chapter), but they are suitable for different scenarios.</p>
<ul>
<li>In algorithm problems, we tend to choose stacks based on arrays because they provide higher operational efficiency and random access capabilities, with the only cost being the need to pre-allocate a certain amount of memory space for the array.</li>
<li>If the data volume is very large, highly dynamic, and the expected size of the stack is difficult to estimate, then a stack based on a linked list is more appropriate. Linked lists can disperse a large amount of data in different parts of the memory and avoid the additional overhead of array expansion.</li>
<li>If the data volume is very large, highly dynamic, and the expected size of the stack is difficult to estimate, then a stack based on a linked list is a better choice. Linked lists can distribute a large amount of data in different parts of the memory and avoid the additional overhead of array expansion.</li>
</ul>
<!-- Source file information -->

12
en/chapter_sorting/sorting_algorithm/index.html
View File

@@ -3584,16 +3584,16 @@
<!-- Page content -->
<h1 id="111-sorting-algorithms">11.1 &nbsp; Sorting algorithms<a class="headerlink" href="#111-sorting-algorithms" title="Permanent link">&para;</a></h1>
<p><u>Sorting algorithms (sorting algorithm)</u> are used to arrange a set of data in a specific order. Sorting algorithms have a wide range of applications because ordered data can usually be searched, analyzed, and processed more efficiently.</p>
<p>As shown in Figure 11-1, the data types in sorting algorithms can be integers, floating point numbers, characters, or strings, etc. Sorting rules can be set according to needs, such as numerical size, character ASCII order, or custom rules.</p>
<p><u>Sorting algorithms</u> are used to arrange a set of data in a specific order. Sorting algorithms have a wide range of applications because ordered data can usually be searched, analyzed, and processed more efficiently.</p>
<p>As shown in Figure 11-1, the data types in sorting algorithms can be integers, floating point numbers, characters, or strings, etc. Sorting criterion can be set according to needs, such as numerical size, character ASCII order, or custom criterion.</p>
<p><a class="glightbox" href="../sorting_algorithm.assets/sorting_examples.png" data-type="image" data-width="100%" data-height="auto" data-desc-position="bottom"><img alt="Data types and comparator examples" class="animation-figure" src="../sorting_algorithm.assets/sorting_examples.png" /></a></p>
<p align="center"> Figure 11-1 &nbsp; Data types and comparator examples </p>
<h2 id="1111-evaluation-dimensions">11.1.1 &nbsp; Evaluation dimensions<a class="headerlink" href="#1111-evaluation-dimensions" title="Permanent link">&para;</a></h2>
<p><strong>Execution efficiency</strong>: We expect the time complexity of sorting algorithms to be as low as possible, with a lower number of overall operations (reduction in the constant factor of time complexity). For large data volumes, execution efficiency is particularly important.</p>
<p><strong>In-place property</strong>: As the name implies, <u>in-place sorting</u> is achieved by directly manipulating the original array, without the need for additional auxiliary arrays, thus saving memory. Generally, in-place sorting involves fewer data movement operations and is faster.</p>
<p><strong>Execution efficiency</strong>: We expect the time complexity of sorting algorithms to be as low as possible, as well as a lower number of overall operations (lowering the constant term of time complexity). For large data volumes, execution efficiency is particularly important.</p>
<p><strong>In-place property</strong>: As the name implies, <u>in-place sorting</u> is achieved by directly manipulating the original array, without the need for additional helper arrays, thus saving memory. Generally, in-place sorting involves fewer data moving operations and is faster.</p>
<p><strong>Stability</strong>: <u>Stable sorting</u> ensures that the relative order of equal elements in the array does not change after sorting.</p>
<p>Stable sorting is a necessary condition for multi-level sorting scenarios. Suppose we have a table storing student information, with the first and second columns being name and age, respectively. In this case, <u>unstable sorting</u> might lead to a loss of orderedness in the input data:</p>
<p>Stable sorting is a necessary condition for multi-key sorting scenarios. Suppose we have a table storing student information, with the first and second columns being name and age, respectively. In this case, <u>unstable sorting</u> might lead to a loss of order in the input data:</p>
<div class="highlight"><pre><span></span><code><a id="__codelineno-0-1" name="__codelineno-0-1" href="#__codelineno-0-1"></a><span class="c1"># Input data is sorted by name</span>
<a id="__codelineno-0-2" name="__codelineno-0-2" href="#__codelineno-0-2"></a><span class="c1"># (name, age)</span>
<a id="__codelineno-0-3" name="__codelineno-0-3" href="#__codelineno-0-3"></a><span class="w"> </span><span class="o">(</span><span class="s1">&#39;A&#39;</span>,<span class="w"> </span><span class="m">19</span><span class="o">)</span>
@@ -3612,7 +3612,7 @@
<a id="__codelineno-0-16" name="__codelineno-0-16" href="#__codelineno-0-16"></a><span class="w"> </span><span class="o">(</span><span class="s1">&#39;E&#39;</span>,<span class="w"> </span><span class="m">23</span><span class="o">)</span>
</code></pre></div>
<p><strong>Adaptability</strong>: <u>Adaptive sorting</u> leverages existing order information within the input data to reduce computational effort, achieving more optimal time efficiency. The best-case time complexity of adaptive sorting algorithms is typically better than their average-case time complexity.</p>
<p><strong>Comparison-based</strong>: <u>Comparison-based sorting</u> relies on comparison operators (<span class="arithmatex">\(&lt;\)</span>, <span class="arithmatex">\(=\)</span>, <span class="arithmatex">\(&gt;\)</span>) to determine the relative order of elements and thus sort the entire array, with the theoretical optimal time complexity being <span class="arithmatex">\(O(n \log n)\)</span>. Meanwhile, <u>non-comparison sorting</u> does not use comparison operators and can achieve a time complexity of <span class="arithmatex">\(O(n)\)</span>, but its versatility is relatively poor.</p>
<p><strong>Comparison or non-comparison-based</strong>: <u>Comparison-based sorting</u> relies on comparison operators (<span class="arithmatex">\(&lt;\)</span>, <span class="arithmatex">\(=\)</span>, <span class="arithmatex">\(&gt;\)</span>) to determine the relative order of elements and thus sort the entire array, with the theoretical optimal time complexity being <span class="arithmatex">\(O(n \log n)\)</span>. Meanwhile, <u>non-comparison sorting</u> does not use comparison operators and can achieve a time complexity of <span class="arithmatex">\(O(n)\)</span>, but its versatility is relatively poor.</p>
<h2 id="1112-ideal-sorting-algorithm">11.1.2 &nbsp; Ideal sorting algorithm<a class="headerlink" href="#1112-ideal-sorting-algorithm" title="Permanent link">&para;</a></h2>
<p><strong>Fast execution, in-place, stable, adaptive, and versatile</strong>. Clearly, no sorting algorithm that combines all these features has been found to date. Therefore, when selecting a sorting algorithm, it is necessary to decide based on the specific characteristics of the data and the requirements of the problem.</p>
<p>Next, we will learn about various sorting algorithms together and analyze the advantages and disadvantages of each based on the above evaluation dimensions.</p>

2
en/search/search_index.json
View File

File diff suppressed because one or more lines are too long

BIN
en/sitemap.xml.gz
View File

Binary file not shown.