我正在寻找一个关于哈希表如何工作的解释-用简单的英语为我这样的傻瓜!
例如,我知道它接受键,计算哈希(我正在寻找一个解释),然后执行某种模运算来计算出它在存储值的数组中的位置,但我的知识到此为止。
谁能解释一下过程吗?
编辑:我并不是特别问哈希码是如何计算的,而是对哈希表如何工作的一般概述。
我正在寻找一个关于哈希表如何工作的解释-用简单的英语为我这样的傻瓜!
例如,我知道它接受键,计算哈希(我正在寻找一个解释),然后执行某种模运算来计算出它在存储值的数组中的位置,但我的知识到此为止。
谁能解释一下过程吗?
编辑:我并不是特别问哈希码是如何计算的,而是对哈希表如何工作的一般概述。
当前回答
这是一个相当深奥的理论领域,但基本轮廓很简单。
本质上,哈希函数只是一个函数,它从一个空间(比如任意长度的字符串)获取内容,并将它们映射到一个用于索引的空间(比如无符号整数)。
如果你只有一个小空间的东西来散列,你可能只需要把这些东西解释为整数,你就完成了(例如4字节字符串)
不过,通常情况下,你的空间要大得多。如果你允许作为键的空间大于你用于索引的空间(你的uint32或其他),那么你不可能为每个键都有唯一的值。当两个或多个东西散列到相同的结果时,您必须以适当的方式处理冗余(这通常被称为冲突,如何处理它或不处理它将略微取决于您使用散列的目的)。
这意味着你不希望得到相同的结果,你也可能希望哈希函数是快速的。
平衡这两个属性(以及其他一些属性)让许多人忙得不可开交!
在实践中,您通常应该能够找到一个已知适合您的应用程序的函数并使用它。
Now to make this work as a hashtable: Imagine you didn't care about memory usage. Then you can create an array as long as your indexing set (all uint32's, for example). As you add something to the table, you hash it's key and look at the array at that index. If there is nothing there, you put your value there. If there is already something there, you add this new entry to a list of things at that address, along with enough information (your original key, or something clever) to find which entry actually belongs to which key.
因此,随着时间的推移,哈希表(数组)中的每个条目要么是空的,要么包含一个条目,要么包含一个条目列表。检索很简单,就像在数组中建立索引,然后返回值,或者遍历值列表并返回正确的值。
当然,在实践中你通常不能这样做,它浪费太多的内存。因此,所有操作都基于稀疏数组(其中唯一的条目是实际使用的条目,其他所有内容都隐式为空)。
有很多方案和技巧可以让它更好地工作,但这是最基本的。
其他回答
哈希表完全基于这样一个事实,即实际计算遵循随机访问机模型,即内存中任何地址的值都可以在O(1)时间或常数时间内访问。
因此,如果我有一个键的宇宙(我可以在应用程序中使用的所有可能的键的集合,例如,滚动no。对于学生来说,如果它是4位,那么这个宇宙就是从1到9999的一组数字),并且一种将它们映射到有限大小的数字集的方法可以在我的系统中分配内存,理论上我的哈希表已经准备好了。
Generally, in applications the size of universe of keys is very large than number of elements I want to add to the hash table(I don't wanna waste a 1 GB memory to hash ,say, 10000 or 100000 integer values because they are 32 bit long in binary reprsentaion). So, we use this hashing. It's sort of a mixing kind of "mathematical" operation, which maps my large universe to a small set of values that I can accomodate in memory. In practical cases, often space of a hash table is of the same "order"(big-O) as the (number of elements *size of each element), So, we don't waste much memory.
现在,一个大集合映射到一个小集合,映射必须是多对一的。因此,不同的键将被分配相同的空间(?? ?不公平)。有几种方法可以解决这个问题,我只知道其中最流行的两种:
Use the space that was to be allocated to the value as a reference to a linked list. This linked list will store one or more values, that come to reside in same slot in many to one mapping. The linked list also contains keys to help someone who comes searching. It's like many people in same apartment, when a delivery-man comes, he goes to the room and asks specifically for the guy. Use a double hash function in an array which gives the same sequence of values every time rather than a single value. When I go to store a value, I see whether the required memory location is free or occupied. If it's free, I can store my value there, if it's occupied I take next value from the sequence and so on until I find a free location and I store my value there. When searching or retreiving the value, I go back on same path as given by the sequence and at each location ask for the vaue if it's there until I find it or search all possible locations in the array.
CLRS的《算法导论》对这个主题提供了非常好的见解。
哈希的计算方式通常不取决于哈希表,而是取决于添加到哈希表中的项。在框架/基类库(如。net和Java)中,每个对象都有一个GetHashCode()(或类似)方法,返回该对象的哈希码。理想的哈希码算法和准确的实现取决于对象中表示的数据。
我的理解是这样的:
这里有一个例子:把整个表想象成一系列的桶。假设您有一个带有字母-数字哈希码的实现,并且每个字母都有一个存储桶。该实现将哈希码以特定字母开头的每个项放入相应的bucket中。
假设你有200个对象,但只有15个对象的哈希码以字母“B”开头。哈希表只需要查找和搜索'B' bucket中的15个对象,而不是所有200个对象。
至于计算哈希码,没有什么神奇的。目标只是让不同的对象返回不同的代码,对于相同的对象返回相同的代码。您可以编写一个类,它总是为所有实例返回相同的整数作为哈希代码,但这实际上会破坏哈希表的用处,因为它只会变成一个巨大的桶。
基本思路
Why do people use dressers to store their clothing? Besides looking trendy and stylish, they have the advantage that every article of clothing has a place where it's supposed to be. If you're looking for a pair of socks, you just check the sock drawer. If you're looking for a shirt, you check the drawer that has your shirts in it. It doesn't matter, when you're looking for socks, how many shirts you have or how many pairs of pants you own, since you don't need to look at them. You just look in the sock drawer and expect to find socks there.
在高层次上,哈希表是一种存储东西的方式,有点像衣服的梳妆台。其基本思想如下:
你有一些可以存储物品的位置(抽屉)。 你想出一些规则,告诉你每件物品属于哪个位置(抽屉)。 当你需要找东西时,你就用这个规则来决定要找哪个抽屉。
这样的系统的优点是,假设您的规则不是太复杂,并且您有适当数量的抽屉,您可以通过查找正确的位置来快速找到您要找的东西。
当你把衣服放好时,你使用的“规则”可能是“袜子放在左边最上面的抽屉里,衬衫放在中间的大抽屉里,等等。”当你存储更抽象的数据时,我们使用一种叫做哈希函数的东西来为我们做这件事。
考虑哈希函数的一种合理方式是将其视为一个黑盒。你把数据放在一边,一个叫做哈希码的数字从另一边出来。从示意图上看,它是这样的:
+---------+
|\| hash |/| --> hash code
data --> |/| function|\|
+---------+
All hash functions are deterministic: if you put the same data into the function multiple times, you'll always get the same value coming out the other side. And a good hash function should look more or less random: small changes to the input data should give wildly different hash codes. For example, the hash codes for the string "pudu" and for the string "kudu" will likely be wildly different from one another. (Then again, it's possible that they're the same. After all, if a hash function's outputs should look more or less random, there's a chance we get the same hash code twice.)
如何构建哈希函数呢?现在,让我们选择“正派的人不应该想太多”。数学家们已经想出了更好和更差的方法来设计哈希函数,但对于我们的目的,我们真的不需要太担心内部。把哈希函数看成是这样的函数就很好了
确定性的(相同的输入给出相同的输出),但是 看起来是随机的(很难预测一个哈希码给出另一个)。
Once we have a hash function, we can build a very simple hash table. We'll make an array of "buckets," which you can think of as being analogous to drawers in our dresser. To store an item in the hash table, we'll compute the hash code of the object and use it as an index in the table, which is analogous to "pick which drawer this item goes in." Then, we put that data item inside the bucket at that index. If that bucket was empty, great! We can put the item there. If that bucket is full, we have some choices of what we can do. A simple approach (called chained hashing) is to treat each bucket as a list of items, the same way that your sock drawer might store multiple socks, and then just add the item to the list at that index.
要在哈希表中查找内容,我们基本上使用相同的过程。我们首先计算要查找的项的哈希代码,它告诉我们要查找哪个桶(抽屉)。如果条目在表中,它就必须在那个bucket中。然后,我们只需查看桶中的所有项,看看我们的项是否在其中。
What's the advantage of doing things this way? Well, assuming we have a large number of buckets, we'd expect that most buckets won't have too many things in them. After all, our hash function kinda sorta ish looks like it has random outputs, so the items are distributed kinda sorta ish evenly across all the buckets. In fact, if we formalize the notion of "our hash function looks kinda random," we can prove that the expected number of items in each bucket is the ratio of the total number of items to the total number of buckets. Therefore, we can find the items we're looking for without having to do too much work.
细节
解释“哈希表”是如何工作的有点棘手,因为哈希表有很多种。下一节将讨论所有哈希表通用的一些通用实现细节,以及不同风格的哈希表如何工作的一些细节。
A first question that comes up is how you turn a hash code into a table slot index. In the above discussion, I just said "use the hash code as an index," but that's actually not a very good idea. In most programming languages, hash codes work out to 32-bit or 64-bit integers, and you aren't going to be able to use those directly as bucket indices. Instead, a common strategy is to make an array of buckets of some size m, compute the (full 32- or 64-bit) hash codes for your items, then mod them by the size of the table to get an index between 0 and m-1, inclusive. The use of modulus works well here because it's decently fast and does a decent job spreading the full range of hash codes across a smaller range.
(这里有时会使用位运算符。如果你的表的大小是2的幂,比如说2k,那么计算哈希码的位与,然后数字2k - 1相当于计算一个模数,而且它明显更快。)
下一个问题是如何选择正确的桶数。如果您选择太多的bucket,那么大多数bucket将是空的或只有很少的元素(对速度有好处-每个bucket只需要检查一些项),但是您将使用大量的空间来简单地存储bucket(不是很好,尽管也许您可以负担得起)。反之亦然——如果存储桶太少,那么每个存储桶平均会有更多的元素,这会使查找时间变长,但会减少内存使用量。
A good compromise is to dynamically change the number of buckets over the lifetime of the hash table. The load factor of a hash table, typically denoted α, is the ratio of the number of elements to the number of buckets. Most hash tables pick some maximum load factor. Once the load factor crosses this limit, the hash table increases its number of slots (say, by doubling), then redistributes the elements from the old table into the new one. This is called rehashing. Assuming the maximum load factor in the table is a constant, this ensures that, assuming you have a good hash function, the expected cost of doing a lookup remains O(1). Insertions now have an amortized expected cost of O(1) because of the cost of periodically rebuilding the table, as is the case with deletions. (Deletions can similarly compact the table if the load factor gets too small.)
哈希策略
到目前为止,我们一直在讨论链式哈希,这是构建哈希表的许多不同策略之一。提醒一下,链式哈希有点像一个服装梳妆台——每个桶(抽屉)可以容纳多个项目,当你进行查找时,你会检查所有这些项目。
然而,这并不是构建哈希表的唯一方法。还有另一类哈希表使用一种叫做开放寻址的策略。开放寻址的基本思想是存储一个槽数组,其中每个槽可以是空的,也可以只保存一项。
In open addressing, when you perform an insertion, as before, you jump to some slot whose index depends on the hash code computed. If that slot is free, great! You put the item there, and you're done. But what if the slot is already full? In that case, you use some secondary strategy to find a different free slot in which to store the item. The most common strategy for doing this uses an approach called linear probing. In linear probing, if the slot you want is already full, you simply shift to the next slot in the table. If that slot is empty, great! You can put the item there. But if that slot is full, you then move to the next slot in the table, etc. (If you hit the end of the table, just wrap back around to the beginning).
Linear probing is a surprisingly fast way to build a hash table. CPU caches are optimized for locality of reference, so memory lookups in adjacent memory locations tend to be much faster than memory lookups in scattered locations. Since a linear probing insertion or deletion works by hitting some array slot and then walking linearly forward, it results in few cache misses and ends up being a lot faster than what the theory normally predicts. (And it happens to be the case that the theory predicts it's going to be very fast!)
Another strategy that's become popular recently is cuckoo hashing. I like to think of cuckoo hashing as the "Frozen" of hash tables. Instead of having one hash table and one hash function, we have two hash tables and two hash functions. Each item can be in exactly one of two places - it's either in the location in the first table given by the first hash function, or it's in the location in the second table given by the second hash function. This means that lookups are worst-case efficient, since you only have to check two spots to see if something is in the table.
Insertions in cuckoo hashing use a different strategy than before. We start off by seeing if either of the two slots that could hold the item are free. If so, great! We just put the item there. But if that doesn't work, then we pick one of the slots, put the item there, and kick out the item that used to be there. That item has to go somewhere, so we try putting it in the other table at the appropriate slot. If that works, great! If not, we kick an item out of that table and try inserting it into the other table. This process continues until everything comes to rest, or we find ourselves trapped in a cycle. (That latter case is rare, and if it happens we have a bunch of options, like "put it in a secondary hash table" or "choose new hash functions and rebuild the tables.")
对于布谷鸟哈希有许多改进的可能,例如使用多个表,让每个槽容纳多个项目,以及制作一个“隐藏”来保存其他地方无法容纳的项目,这是一个活跃的研究领域!
Then there are hybrid approaches. Hopscotch hashing is a mix between open addressing and chained hashing that can be thought of as taking a chained hash table and storing each item in each bucket in a slot near where the item wants to go. This strategy plays well with multithreading. The Swiss table uses the fact that some processors can perform multiple operations in parallel with a single instruction to speed up a linear probing table. Extendible hashing is designed for databases and file systems and uses a mix of a trie and a chained hash table to dynamically increase bucket sizes as individual buckets get loaded. Robin Hood hashing is a variant of linear probing in which items can be moved after being inserted to reduce the variance in how far from home each element can live.
进一步的阅读
有关哈希表基础知识的更多信息,请查看关于链式哈希的讲座幻灯片以及关于线性探测和罗宾汉哈希的后续幻灯片。你可以在这里学到更多关于布谷鸟哈希的知识,以及哈希函数的理论性质。
你取一堆东西,和一个数组。
对于每一个东西,你为它建立一个索引,称为哈希。关于哈希的重要事情是它“分散”了很多;你不希望两个相似的东西有相似的哈希值。
你把东西放到数组中哈希值表示的位置。在一个给定的哈希中可以有多个对象,所以你可以将这些对象存储在数组或其他合适的东西中,我们通常称之为bucket。
当你在哈希中查找东西时,你会经历相同的步骤,计算哈希值,然后查看那个位置的bucket中有什么,并检查它是否是你要寻找的东西。
当你的哈希工作得很好并且你的数组足够大时,在数组的任何特定下标处最多只会有很少的东西,所以你不需要看太多。
额外的好处是,当你的哈希表被访问时,它会把找到的东西(如果有的话)移动到桶的开头,这样下次它就会是第一个被检查的东西。