Bandit Algorithms for Website Optimization
John Myles White
Language: English
Pages: 88
ISBN: 1449341330
Format: PDF / Kindle (mobi) / ePub
When looking for ways to improve your website, how do you decide which changes to make? And which changes to keep? This concise book shows you how to use Multiarmed Bandit algorithms to measure the real-world value of any modifications you make to your site. Author John Myles White shows you how this powerful class of algorithms can help you boost website traffic, convert visitors to customers, and increase many other measures of success.
This is the first developer-focused book on bandit algorithms, which were previously described only in research papers. You’ll quickly learn the benefits of several simple algorithms—including the epsilon-Greedy, Softmax, and Upper Confidence Bound (UCB) algorithms—by working through code examples written in Python, which you can easily adapt for deployment on your own website.
- Learn the basics of A/B testing—and recognize when it’s better to use bandit algorithms
- Develop a unit testing framework for debugging bandit algorithms
- Get additional code examples written in Julia, Ruby, and JavaScript with supplemental online materials
The Agrarian Question in the Neoliberal Era: Primitive Accumulation and the Peasantry
code, this proportional approach would look like: def categorical_draw(probs): z = random.random() cum_prob = 0.0 for i in range(len(probs)): prob = probs[i] cum_prob += prob if cum_prob > z: return i return len(probs) - 1 def select_arm(self): z = sum(self.values) probs = [v / z for v in self.values] return categorical_draw(probs) In practice, this very naive algorithm isn’t something people actually use. To reconstruct the algorithm people actually use, we need to make two changes to it.
select_arm(self): z = sum([math.exp(v / self.temperature) for v in self.values]) probs = [math.exp(v / self.temperature) / z for v in self.values] return categorical_draw(probs) def update(self, chosen_arm, reward): self.counts[chosen_arm] = self.counts[chosen_arm] + 1 n = self.counts[chosen_arm] value = self.values[chosen_arm] new_value = ((n - 1) / float(n)) * value + (1 / float(n)) * reward self.values[chosen_arm] = new_value return Now that we have the Softmax algorithm fully described and
programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing a CD-ROM of examples from O'™Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into
programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing a CD-ROM of examples from O'™Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into
you deploy a bandit algorithm. The algorithms we’ve presented will not handle most sorts of change in the underlying values of arms well. The problem has to do with the way that we estimate the value of an arm. We typically updated our estimates using the following snippet of code: new_value = ((n - 1) / float(n)) * value + (1 / float(n)) * reward self.values[chosen_arm] = new_value Intelligent Initialization of Values | 63 The problem with this update rule is that 1 / float(n) goes to 0 as