Category Archives: Uncategorized

Rewards: The Best Choice for Brands Building Reciprocal Loyalty

Long-lasting loyalty is reciprocal, where both brands and consumers benefit from the relationship. But what fosters reciprocal loyalty? What can a brand offer its consumers to cultivate the type of two-way relationship that will keep them coming back time and time again?

When loyalty is reciprocal, rewards become the most important way to bridge the gap between a brand and its consumers. BigDoor is leading the charge on the concept of reciprocal loyalty, and today we want to help your next loyalty campaign become mutually beneficial. This post walks you through the basics of the types of points and currency that back rewards; outlines three main types of rewards brands offer customers; provides examples on how you can implement rewards into your next loyalty campaign; and offers advice about which reward(s) is the best choice for your audience.

There are two main perspectives when it comes to building reciprocal loyalty through rewards: the brand side, and the consumer side. We’ll talk about rewards from the brand perspective in this post, and will cover the consumer perspective in a post that’s coming soon.

Let’s dive in!

Points & Currency: Dollar-backed vs. Non Dollar-backed

Dollar-backed currency is what all traditional rewards programs are built on. With dollar-backed currencies, the financial liability of the reward occurs when a consumer earns the points, not when the consumer redeems those points. A perfect example is an airline mileage program. A consumer buys a ticket, flies 500 miles, and they get 500 mileage points that they can redeem at a future date for a flight, an upgrade, or a discount. While the points they are receiving may not have a true cash value, they do have a real cost to the airline. As a result, the airline needs to carefully account for how many points are outstanding, and their associated financial liability. This means that the airline can only afford to grant points when a consumer is making a purchase and spending real dollars with the airline.

Non dollar-backed currency is a new concept being pioneered by BigDoor (we’ll be writing more about it soon), and provides significantly more flexibility than traditional dollar-backed currency. With non dollar-backed programs, the financial liability of the rewards occurs at the point of reward redemption, not at the point of earning. Any rewards that have underlying costs associated with them must have inventory and time limitations, and non dollar-backed rewards provide an opportunity for no inventory restrictions at all as they do not have direct cost associated with redemption. This structure allows a brand to set a pre-determined monthly or annual rewards budget, and whether the consumers earn a million points or ten million points, the pre-determined budget for the brand stays the same. The consumer can then be rewarded for a number of what we call “soft actions” like visiting a site, referring a friend, following the brand on Twitter, watching a video, or engaging in content. Consumers can also earn points for making purchases with non dollar-backed rewards, as well.

Dollar-backed and non dollar-backed currency support different types of rewards, but both directly effect the value of the reward a brand is offering. The quadrant below breaks down the value of a reward based on the two points that spur a brand to choose the type of currency backing the reward: brand affinity and cost.

Screen Shot 2013-10-21 at 1.43.51 PM

Brand affinity is the feeling of goodwill a customer or community member holds about a brand. When a company’s brand affinity is high, consumers are feeling good about the products and/or services they company provides. When  brand affinity is low, consumers are unsure about the goods, or message behind the goods, that they are paying for. Cost can be thought of as “incremental cost,” where the cost to a business is higher or lower depending on the cost of the units of output processed.

When analyzing how brand affinity and cost work together to yield the value of rewards, we see four possibilities: decreasing brand affinity with high incremental costs (i.e. using a dollar-backed campaign to send cheaply made mugs to a large volume of consumers); decreasing brand affinity with no incremental costs (i.e. using a non dollar-backed campaign to send an off-brand message to an online community); increasing brand affinity with high incremental costs (i.e. using a dollar-backed campaign to ship high-priced swag bags to consumers as rewards); and increasing brand affinity with no incremental costs (i.e. using a non dollar-backed campaign to encourage users to interact with your online content for points).

Out of the four choices displayed in the quadrant, it’s best to aim for the sweet spot of increasing brand affinity with no incremental costs. A reward that increases brand affinity and lowers costs is the most cost-effective and consumer-pleasing value that any reward can offer reciprocally; the consumer is happy with the reward and continues to love the brand, and the brand is happy that costs are low and that the consumer is pleased. This is where dollar-backed or non-dollar backed currency comes in.

The grid speaks directly to brands deciding whether their rewards should be supported by dollar-backed or non dollar-backed currency. Both options yield a variety of possibility for the types of rewards a brand can offer, but when a brand is supported to use non dollar-backed currency, they gain a huge competitive advantage.

Through non dollar-backed currency, brands can “pay” users for completing “soft” actions, like gaining points for interacting with the brand through social channels, commenting on and sharing content, providing feedback on how to improve the customer experience, and more. If you’re giving users the opportunity to rack in points or coins only for dollar-backed purchases, you are offering a limited experience to increase brand affinity, and run into a problem. In today’s world, so much of what builds brand affinity — and increases loyalty — occurs outside of the traditional pay-to-purchase model, and those actions need to be rewarded. Even though you may not be rewarding users for soft actions, you can bet your competitors are – or will be very soon.

Another advantage of non dollar-backed currency is that a brand can calculate exactly how much they’re going to spend on a rewards program or campaign before it even begins, regardless of user behavior. In traditional dollar-backed transactions, the cost of rewards rests solely on consumer redemption (i.e. the number of people who redeem the reward drives the amount of swag that needs to be manufactured, stored, and shipped). Non dollar-backed currency allows you to set your budgets outside of you customers redemption cycle, which allows for cutting unnecessary costs out of your budget and for your accounting accrual to remain steady.

Rewards: Dollar-backed  vs. Non Dollar-backed

Dollar-backed rewards have an underlying cost to the brand (i.e. discounts or a free airline flight). Non dollar-backed reward programs may include some rewards that have an underlying cost, but each of those rewards needs to have limited overall inventory, and then also provide rewards that have no underlying cost to the brand that have no inventory restrictions (i.e. sweepstakes, exclusive content access, or trial versions of products). Just like non dollar-backed currency, there is an advantage to being able to offer non dollar-backed rewards.

With traditional dollar-backed rewards, companies tend face high upfront costs without knowing the full mechanics of the program. For example, brands have to account for the cost of storing the dollar-backed rewards in a warehouse, paying for shipping, and setting a logistics process in place to deliver the rewards to the consumers.

Screen Shot 2013-10-23 at 10.32.30 AM

On the consumer side, dollar-backed rewards don’t yield immediate results, and the “instant gratification” factor that drives return visitors to complete incentivized actions gets swept away. This lack of engagement, coupled with internal uncertainty around costs and fulfillment, can be a pain in the you-know-what for your accounting, marketing, and legal teams. With non dollar-backed rewards, you can take back control over your rewards budget and put your team at ease.

However, there is definitely a time and place for dollar-backed rewards that can skyrocket user engagement for a chance to redeem. Let’s go back to the example of frequent flier miles programs. Frequent flier miles are always dollar-backed as they are rewarded only after a transaction has been made, but airline consumers LOVE their frequent flier miles and will complete unprecedented actions to keep and grow their milage. You know that friend you have who takes three or four weekend trips near the end of the year just to keep her gold status on her favorite airline? She’s not alone. In January 2006, the US alone boasted over 120 million customers enrolled in frequent flier miles programs. Airlines have found the golden ticket through dollar-backed rewards.

Screen Shot 2013-10-23 at 11.20.04 AM

Three Types of Rewards

Now that you know the two ways to back rewards, it’s time to decide what type of rewards to offer your consumers. Just like customers, rewards come in all shapes and sizes. Despite the endless possibilities of specific rewards brands can offer their customers, there are three main categories that all rewards fall into. Here we go!

1. Sweepstakes (a BigDoor must-have)

A sweepstakes is an opportunity given to a consumer by a brand to enter for a chance to win a prize. The most common sweepstakes offer experiential rewards, where consumers can collect points by completing incentivized actions in order to enter a chance to win a trip, instant-win giveaway, or product. At BigDoor, we see sweepstakes as a key reward to have in your loyalty campaign arsenal, and encourage our customers to implement them into their loyalty campaign strategy.

Why we love sweepstakes:

Instant gratification. While lotteries and sweepstakes are very different mechanically (and legally), the human psychology of participating in both is very similar. Purchasing a lottery ticket is instantly gratifying for participants, regardless of whether or not they win. Lotto participants create an experience out of purchasing a ticket; they watch the “pot” until it reaches certain levels, eagerly rush to grocery stores or gas stations to purchase their tickets, and even go so far as to strategize their purchases and map out where previous winners bought tickets. Part of the fun of entering is receiving the ticket. This frenzy is fueled by the aspirational opportunity to win an incredible prize (just check out these folks) that speaks directly to the consumer psyche. Harnessing the energy sweepstakes provide to power your customer engagement yields incredible results, with entirely predictable financial costs.

Screen Shot 2013-10-23 at 11.27.54 AM Courtesy of The Guardian.

Calculated reward cost. Sweepstakes are monitored rewards that can be planned down to the cent, which is incredibly cost-effective for your business. As a brand, you will only spend as much as the sweepstake is worth, without any hidden fees or logistics associated with redemption. You can alter the cost for each sweepstakes as you push the campaign live, which means this reward can match the flux of your brand’s cash flow at any given time. Talk about a win-win!

Awesome prizes. When you have total control over the cost, you can craft seriously awesome rewards based on your budget. Sweepstakes are typically only rewarded to one person at a time, so instead of spending $10K on a batch of coffee mugs to ship you, you can spend $10K on sending one lucky winner on their dream trip to interact with your brand in some capacity. Incredible prizes entice users to complete the actions allowing them to enter, which directly affects user engagement and behavior.

Flexible entry timeframes. Whether you’d like your sweepstakes to run for one day or three months, any entry timeframe is possible. Before you set your entry parameters, make sure you consider the goal of your sweepstakes. For example, if you’re giving away a last-minute trip to see the finale of a show, you might want to shorten the timeframe for entry. If your goal is to send users on quests in order to earn points to enter the sweepstakes, you might want to increase the timeframe for entry so that a large amount of users are eligible to win. You should judge the timeframe for your sweepstakes based on the results you’d like to receive from participants.

The downside:

Sweepstakes are only as good as you make them. This goes without saying, but a sweepstakes will only give you what you give it. If you’re a fashion retailer who is planning to send one lucky sweepstakes winner to Maui, you could advertise the trip as “Win a trip to Maui!” and see a mediocre response. On the other hand, you could send your winner on a trip to Maui where they will get the chance to meet with one of your designers and talk fashion, and then enjoy a $500 shopping spree at one of your stores. When it comes to sweepstakes, the devil is in the details. Put in what you want to get back.

2. Digital (a BigDoor must-have)

Digital rewards are any reward that is redeemed online (i.e. social media shout outs, badges, points, coins, etc.). Digital rewards are the newest — and fastest growing — category of rewards as retailers and publishers move further into the digital experience when interacting with their consumers. BigDoor sees the future of rewards living in this category, and considers the option of digital rewards a must-have for all brands to want to dominate their markets over the next few years.

Why we love digital rewards:

Bridge the gap. Digital rewards are one of the best ways to bridge the gap between a brand and a consumer. More than ever before, consumers are reaching out to brands where they expect to get immediate responses: online. Experiential rewards that are delivered digitally, like Twitter shout outs or points for interacting with online content, offer a connection between a brand and consumer that is new, exciting, and increasingly expected from your brand’s loyalist fans.

Screen Shot 2013-10-23 at 11.33.54 AM

Cost-effective. When a brand offers digital rewards, they have total control over how much the campaign will cost the company. Digital campaigns are easy to turn on and off depending on audience response, with few costs associated with pulling the plug early. Digital rewards are also some of the lowest costing rewards available; many times, they’re even free. Outside of paying employees to deliver digital rewards through social media and running loyalty campaigns (don’t have one of those? Check our our product!), digital rewards can be created to fit even the tightest budgets. No other type of reward can ensure such a controlled environment.

Instant delivery. Through digital rewards, brands can respond to consumers and provide rewards instantly. Rather than shipping a product as a reward, or opening a waiting period for people to enter to win a sweepstakes, digital rewards can be awarded as soon as the incentivized action has taken place. This type of reward instantly delights consumers, driving engagement and fostering loyalty through positive interactions with a brand. For example, this year Starbucks used a digital, non dollar-backed rewards campaign to entice buyers to purchase bags of coffee from their local grocery stores. When a consumer purchased a bag and entered the promotional code on the Starbucks website, they were instantly rewarded by collecting Stars, which encouraged them to buy more coffee until they had enough stars to reach their goal and receive a gift card (which was also redeemable digitally). Digital rewards are a perfect way for retailers to award users instantly for purchases.

Endless possibilities. In the digital world, the possibilities of reward are as endless as the interwebs itself 🙂 Leveraging existing platforms, experimenting with online communities, and even directly asking your consumers what they want as a reward make these rewards incredibly relevant for any brand. The sky is the limit, and we’re just getting started with harnessing the potential of digital rewards.

The downside:

Not everything can be delivered digitally. Even today, there are still some rewards that are better delivered in person. Until we find a way to pixelize limited-edition, branded swag, we’ll have to stick with snail mail every now and again. Better start investing in Wonka Vision.

Large online communities yield the best results. Digital rewards typically do best when they’re presented to large online communities, which means the more you put into building and growing your digital presence, the more digital rewards will help build loyalty. If your consumers and fans aren’t online, digital rewards probably won’t be right for your business. Targeting your consumers where they already are is still the best approach to rewards.

3. Tangible

Tangible rewards are the physical products given to consumers when they complete incentivized tasks (i.e. completing on-site quests, redeeming a set number of coins, etc.). Here at BigDoor, we believe that the future of rewards lives most heavily in the first two categories (sweepstakes and digital), but tangible rewards still leave a strong mark on the rewards categories landscape. They’re here to stay for the time being, and should be considered whenever a brand puts rewards together for their loyalty program.

Why we love tangible rewards:

Novel, rare merchandise. Tangible rewards are the quickest way to deliver limited-edition merchandise or memorabilia. Novel merchandise can still be considered experiential because the customer is offered the opportunity to have a fantastic experience with the reward; receiving a special piece of merchandise can be a fantastic experience depending on delivery, packaging, and branding. For example, a baseball hat signed by a baseball player is a great piece of unique merchandise to reward consumers with.

Branded rewards. Tangible rewards also offer an opportunity for a brand to engage consumers in a forward-facing way. Building off of the example above, a branded baseball hat signed by a famous player delivers a positive experience, but also ties the consumer directly back to your brand, increasing brand affinity and building loyalty. Just be sure to give some serious thought onto what your brand slaps its logo on before sending it out as a reward. Many a-failed marketing campaigns have occurred when branding goes horribly wrong.

Screen Shot 2013-10-23 at 11.41.11 AM

The downside:

Tangible isn’t sustainable. Fulfillment costs, paying humans to pack things, and figuring out shipping logistics; this process is limiting, and is just not sustainable for many businesses. Rewards programs should include items outside of physical goods as tangible items are not always the best way to boost loyalty. Even though there might be added value to the brand and the program through tangible rewards, the cost to the brand might not be worth it.

Lackluster products. When brands consider giving away tangible rewards, they tend to be on the cheaper side as they are mass-produced. These rewards do not entice consumers as much as an experiential or state-of-the-art reward would. The selection of these rewards could be due to the fact that the brand has to find a quick way to get in front of a large group, but the brainstorming behind tangible rewards tends to be a little dismal. The next time you even consider giving away that free coffee mug, stop yourself before you end up with a warehouse that looks like this:

Screen Shot 2013-10-23 at 11.52.35 AM

And your customers’ homes start looking like this:

Screen Shot 2013-10-23 at 11.44.35 AM Courtesy of The Daily Herald.

Physical products don’t drive loyalty. Time and time again, studies prove that physical rewards don’t have desired effects. Consumers choose experiences with brands, and as we continue to move into the digital world, digital rewards and experiential rewards really drive it home. Tangible items might drive brand affinity for a short amount of time, but when it comes to driving lasting loyalty, consider yourself pushed to the bottom of the pile — much like that $5 t-shirt you sent out last month.

So, What Rewards Should I Offer?

Brands have diverse goals when it comes to selecting rewards to offer through a loyalty program, but one thing is always certain: brands want rewards to delight their consumers.

It’s best to work backwards from your consumer when selecting which type of reward your brand will offer. What entices your audience the most when completing incentivized tasks? Does your community have a large social following that you can leverage when introducing new rewards? Have you polled consumers to see what they’d be most excited to receive? Study your users’ habits, and offer rewards accordingly. If you haven’t spent much time figuring out what specifically drives your users to engage, feel free to reach out to us and we’d be happy to provide you with more of our insights and learnings.

At BigDoor, we see the future of loyalty programs being forever changed by both non dollar-backed rewards that serve to increase brand affinity. However, a truly complete loyalty program allows for flexibility across all available rewards. The best rewards programs provide the brand with predictable and controllable costs, while giving their consumers flexibility in redeeming rewards that serve to increase brand affinity. This is the sweet spot of reciprocal loyalty.


Now that you know the in’s and out’s of selecting the right reward for your program, we’d love to hear from you. What types of rewards does your brand offer? Do you find that one (or two, maybe all three!) category works best? Leave your thoughts in the comments below!

If you’d like to hear more about our thoughts on rewards, stay tuned for a continued series on the blog. If you’re inspired to continue the conversation, drop us a line any time at
team@bigdoor.com
, and one of our loyalty specialists will provide you with all the information you need.

BigDoor Goes Big with Performance Upgrades

As I write this, 50MM monthly active users are engaging with the BigDoor platform across our publisher sites. For a company that started with the simple idea of making online experiences more fun and engaging, knowing that our platform reaches that many people is astounding. As we launch more and more gamified loyalty programs with larger audiences, the importance of our platform’s performance and scalability has increased as well. To stay ahead of our publisher’s needs, we’ve been making major updates to our tech stack, ensuring that our platform comfortably supports the 20MM user actions per day and 1.5MM users who access BigDoor programs during peak hours.

Knowing that our platform integrates with the websites of some of the most influential brands across multiple industries might keep some of us up at night, but it also drives us to constantly push the envelope in terms of what our clients can expect in terms of features and performance.

In the past six months, we implemented significant performance improvements resulting in an average API latency of less than 250ms.

Visitors to any of our publisher’s sites continue to consistently and reliably enjoy our end-user experience. We’ve met 100% of our SLA’s high availability requirements this year, exceeding our 99.9% uptime goals for every month in Q1.

But, we are overachievers and we aren’t done yet.

On Wednesday night, we notified our publishers and Twitter followers that the BigDoor platform would be temporarily unavailable for an hour of maintenance. The maintenance window was required for the deployment of a new version of our backend database sharding framework, which will allow us to support even higher availability, increased responsiveness and more scaling capacity for our ever-increasing audience sizes.

The updates to our backend systems have only been in effect for a short time, but we are pleased with the results they are showing already.

Announcing BigDoor’s Newest Platform Feature: Internationalization

In response to a growing number of current and future partner’s inquiries and suggestions, we are very excited to announce BigDoor’s newest platform feature: Internationalization.

Working with brands that achieve global reach, we acknowledge the need for rewards and loyalty programs that engage customers internationally and across languages. Unlike other vendors in the space, touting their “global” or “world leading” platforms with English-only language support, BigDoor has always prided ourselves in making our platform available in any language, but we weren’t satisfied with the complex translation progress that slowed down our typical implementation process. We decided to launch internationalization to enable brands around the world to easily implement a BigDoor Gamified Loyalty Program in any language (or combination of languages).

With internationalization the BigDoor user interface will reflect browser-specified  languages allowing visitors to seamlessly interact with a brand’s loyalty program, no matter what language version of the site they are on. Badges, quests, rewards and achievements will display in the browser-specified language throughout a customer’s session. The feature will also cover BigDoor powered redemption emails, ensuring that the entire BigDoor experience is available to customers, no matter what language they are using.

BigDoor publishers will be able to add language configurations to current loyalty programs, without affecting the existing experience. The internationalization feature will also allow publishers to break down existing analytics by language, giving even deeper access to valuable customer data and insights.

To learn more about BigDoor’s internationalization feature, or to talk with a BigDoor Loyalty Expert, please contact us at: Info@BigDoor.com.

Happy Halloween From All Of Us @BigDoor

Happy Halloween!

While we believe that it is always acceptable to dress up as your favorite thing, Halloween is the day where you don’t have to explain! From all of us at BigDoor, we wish everyone a fun and safe Halloween. For those in the Storm Sandy affected areas, who might be postponing Halloween, we are thinking of you and hope you enjoy the festivities soon.


If you want to check out some of the photos from last years Halloween, you can check them out here.

BigDoor’s Jeff Malek Talks Databases and Horizontal Scaling

When talking to people about BigDoor, we often get asked about our backend database systems. Potential clients want to know whether or not our system can handle their traffic (yes, it can) and other start-ups want to know how we scaled to handle the high traffic levels of our awesome clients. We thought it would be useful to give a little bit more information about our system as well as share some of the things we learned in the process. There is no better expert in this topic than our CTO/Co-founder Jeff Malek. He took a break from working his daily magic to give some perspective and advice. 

A friend was recently asking about our backend database systems.  BigDoor’s  systems are able to successfully handle high-volume transactional traffic through our API coming from various customers, having vastly different spiking patterns,  including traffic from a site that’s in the top-100 list for highest traffic on the net.   Don’t get me wrong; I don’t want to sound overly impressed with what our little team has been able to accomplish, we’re not perfect by any means and we’re not talking about Google or FaceBook traffic levels.  But serving requests to over one million unique users in an hour, and doing 50K database queries per second isn’t trivial, either.

I responded to my friend along the following lines:

  1. If you’re going with an RDBMS, MySQL is the right, best choice in my opinion.  It’s worked very well for us over the years.
  2. Since you’re going the standard SQL route:
    1. If your database is expected to grow in step with traffic, and you’re thinking about sharding early – kudos.  You’re likely going to have to do it, sooner or later.
      1. Sooner vs. later if you’re in the cloud and running under its performance constraints.
      2. Do it out of the gate, if you have time, after you’ve figured out how you’re going to do it (i.e. whether you’re going to leverage a tool, DYI, etc).
        1. In other words, if you have time, don’t “see how long you can survive, scaling vertically”.
          1. Sharding while running the race : not a fun transition to make.
      3. I realize what I’m saying is counter to popular thinking, which is “don’t shard unless you absolutely have to”.
        1.  Without the assumption that your data set is going to grow in step with your traffic, I’d be saying the same thing.
    2. Designing your schema and app layer for sharding, sharded on as few keys as possible, ideally just one, is not future-proofing, it’s a critical P0.
  3. Since you’re going to be sharding MySQL, your options are relatively limited last I checked.
    1. Ask for input from folks who have done it before.
    2. The other sharding options I started looking at over two years ago all had disallowing limitations, given our business model.
    3. At quick search-glance just now, it also does appear that dbshards is ruling this space at this point.
  4. So barring any other options I’m missing, your best options that I’m aware of:
    1. dbshards
      1. Definitions we/they use, to help clarify discussion  :
        1. global tables : tables that contain the same data on every shard, consistency managed by dbshards.
        2. shard : two (primary and secondary) or more hosts that house all global table data, plus any shard-specific data.
        3. shard tree : conceptually, the distribution of sharded data amongst nodes, based on one or more shard keys.
        4. reliable replication : dbshards-proprietary replication, more details on this below.
      2. pros
        1. The obvious : you’ll be able to do shard-count more reads and writes that you’d otherwise be able to do with a monolithic, non-sharded backend (approximately).
          1. Alternatively, with a single-primary read-write or write-only node, and multi-secondary read-only nodes you could scale reads to some degree.
            1. But be prepared to manage the complexities that come along with eventual read-consistency, including replication-lag instrumentation and discovery, beyond any user notifications around data not being up-to-date (if needed).
        2. It was built by folks who have only been thinking about sharding and its complexities, for many years
          1. who have plans on their roadmap to fill any gaps with their current product
            1. gaps that will start to appear quickly, to anyone trying to build their own sharding solution.
              1. In other words, do-it-yourself-ers will at some point be losing a race with CodeFutures to close the same gaps, while already trying to win the race against their market competitors.
        3. It’s in Java, vs. some other non-performant or obscure (syntactically or otherwise) language.
        4. It allows for multiple shard trees; if you want (or have to) trade in other benefits for sharding on more than one key, you can.
          1. Benefits of just sharding on one key include, amongst other things, knowing that if you have 16 shards, and one is unavailable, and the rest of the cluster is available, 1/16th of your data is unavailable.
            1. With more than one shard tree, good luck doing that kind of math.
        5. It provides a solution for the auto-increment or “I need unique key IDs” problem.
        6. It provides a solution for the “I need connection pooling that’s balanced to shard and node count” problem.
        7. It provides a solution for the “I want an algorithm for balancing shard reads and writes”.
          1. Additionally, “I want the shard key to be based on a column I’m populating with the concatenated result of two other string keys”.
        8. It has a distributed-agent architecture, vs. being deeply embedded (e.g. there are free-standing data streaming agents, replication agents, etc instead of MySQL plugins, code modules, etc ).
          1. Provides future-proofing, scaleability and plug-ability.
          2. Easier to manage than other design approaches.
        9. Streaming agents allow you to plug into the update/insert stream, and do what you like with changes to data.
          1. We use this to stream data into Redis, amongst other things.  Redis has worked out very well for us thus far, by the way.
          2. Other dbshards customers use this to replicate to other DBMS engines, managed by dbShards or not, such as a column store like MonetDb, InfoBright, even a single standalone MySQL server if it can handle the load.
        10. It supports consistent writes to global tables; when a write is done to a global table, its guaranteed to have been done on all global tables.
        11. It doesn’t rely on MySQL’s replication and its shortcomings, but rather on its own robust, low-maintenance and flexible replication model.
        12. Its command-line console provides a lot of functionality you’d rather not have to build.
          1. Allows you to run queries against the shard cluster, like you were at the MySQL command line.
          2. Soon they’re releasing a new plug-compatible version of the open source MyOSP driver, so we’ll be able to use the same mysql command line to access both dbShards and non-dbShards managed MySQL databases.
        13. Its web console provides a lot of functionality you’d rather not have to build.
          1. Agent management and reporting, including replication statistics.
          2. Displays warning, error, diagnostic information, and graphs query counts with types.
          3. Done via the “dbsmanage” host, which provides centralized shard node management as well.
        14. It’s designed with HA in mind.
          1. Each shard is two (or optionally more, I think) nodes.  We put all primary nodes in one AWS availability zone, secondaries in a different one, for protection against zone outages.
          2. Write consistency to two nodes; in other words DB transactions only complete after disk writes have completed on both nodes.  Secondary writes only require file-system (vs. MySQL) disk writes.
          3. Managed backups with configurable intervals; MySQL EC2/EBS backups aren’t trivial.
          4. Web-console based fail-over from primary to secondary; this is very helpful, particularly for maintenance purposes.
        15. Proven to work well in production, by us and others.
          1. We’ve performed 100K queries per second in load-testing, on AWS/EC2, using m1.xlarge instances.
        16. Designed with the cloud and AWS in mind, which was a great fit for us since we’re 100% in AWS.
        17. “dbsmanage” host
        18. Drivers included, of course.
          1. In addition to MyOSP, they have JDBC, PQOSP (native Postgres), ADO OSP (for .NET), and soon ODBC.
        19. Go-fish queries allow you to write standard SQL against sharded data
          1. e.g. sharded on user.id : SELECT * FROM user where FirstName=’Foo’;
            1. will return all results from all shards performing automatic aggregation
              1. sorting using a streaming map-reduce method
        20. Relatively easy to implement and go live with; took us about six weeks of hard work, deadline-looming.
        21. It’s the market-leading product, from what I can tell.
          1. 5 of the Top 50 Facebook apps in the world run dbShards.
        22. It supports sharding RDBMSs besides MySQL, including Postgres, DB2, SQL Server, MonetDb, others coming.
        23. Team : top-notch, jump-through-their-butts-for-you, good guys. 
        24. Ability to stream data to a highly performant BI backend.
      3. cons
        1. As you can see, some of these are on the pro list too, double-edged swords.
        2. Cost – it’s not free obviously, nor is it open source.
          1. Weigh the cost against market opportunity, and/or the additional headcount required to take a different approach.
        3. It’s in Java, vs. Python (snark).
        4. Doesn’t rely on MySQL replication, which has its annoyances but has been under development for a long time.
          1. Nor is there enough instrumentation around lag.  What’s needed is a programmatic way to find this out.
        5. Allows for multiple shard trees.
          1. I’m told many businesses need this as a P0, and that might be true, even for us.
          2. But I’d personally prefer to jump through fire in order to have a single shard tree, if at all possible.
            1. The complexities of multiple shard trees, particularly when it comes to HA, are too expensive to justify unless absolutely necessary, in my humble opinion.
        6. Better monitoring instrumentation is needed, ideally we’d have a programmatic way to determine various states and metrics.
        7. Command line console needs improvement, not all standard SQL is supported.
          1. That said, we’ve managed to get by with it, only occasionally using it for diagnostics.
        8. Can’t do SQL JOINs from between shard trees.  I’ve heard this is coming in a future release.
          1. This can be a real PITA, but it’s a relatively complex feature.
          2. Another reason not to have multiple shard trees, if you can avoid them.
        9. Go-fish queries are very expensive, and can slow performance to a halt, across the board.
          1. We’re currently testing a hot-fix that makes this much less severe.
          2. But slow queries can take down MySQL (e.g. thread starvation), sharding or no.
        10. HA limitations, gaps that are on their near-term roadmap, I think to be released this year:
          1. No support for eventually-consistent writes to global tables means all primaries must be available for global writes.
            1. Async, eventually consistent writes should be available as a feature in their next build, by early October.
          2. Fail-over to secondaries or back to primaries can only happen if both nodes are responding.
            1. in other words, you can’t say via the console:
              1. ‘ignore the unresponsive primary, go ahead and use the secondary’
            2. or:
              1. ‘stand me up a new EC2 instance for a secondary, in this zone/region, sync it with the existing primary, and go back into production with it’
          3. Reliable replication currently requires two nodes to be available.
            1. In other words, if a single host goes down, writes for its shard are disallowed.
              1. In the latest versions, there’s a configuration “switch” that allows for failing-down to primary
                1. But not fail down to secondary.  This is expected in an early Q4 2012 version release.
          4. dbsmanage host must be available.
            1. dbshards can run without it or a bit, but stats/alerts will be unavailable for that period.
          5. Shard 1 must be available for new auto-increment batch requests.
          6. go-fish queries depend on all primaries (or maybe all secondaries via configuration, but not some mix of the two as far as I’m aware) to be available
    2. DYI
      1. I can rattle off the names of a number of companies who have done this, and it took many months longer than our deployment of dbshards (about six weeks, largely due to the schema being largely ready for it).
      2. Given a lot of time to do it, appeals to me even now, but I still wouldn’t go this route, given the pros/cons above.
    3. The latest release of MySQL Cluster may be an option for you, it wasn’t for us back with MySQL 5.0, and not likely now, due to its limitations (e.g. no InnoDB).
    4. AWS RDS was an option for us from the onset, and I chose to manage our own instances running MySQL, before deciding how we’d shard.
      1. For the following reasons:
        1. <li>I wanted ownership/control around the replication stream, which RDS doesn’t allow for (last I looked) for things like:
          1. BI/reporting tools that don’t require queries to be run against secondary hosts.
            1. This hasn’t panned out as planned, but could still be implemented, and I’m happy we have this option, hope to get to it sometime soon.
          2. Asynchronous post-transaction data processing.
            1. This has worked out very well, particularly with dbshards, which allows you to build streaming plugins and do whatever you want when data changes, with that data.
              1. Event-driven model.
              2. Better for us than doing it at the app layer, which would increase latencies to our API.
        2. Concern that the critical foundational knobs and levers would be out of our reach.
          1. Can’t say for sure, but this has likely been a good choice for our particular use-case; without question we’ve been able to see and pull levers that we otherwise wouldn’t have been able to, in some cases saving our bacon.
        3. Their uptime SLAs, which hinted at unacceptable downtime for our use-case.
          1. Perhaps the biggest win on the decision not to use RDS; they’ve had a lot of down-time with this service.
        4. Ability to run tools, like mk-archiver (which we use extensively for data store size management), on a regular basis without a hitch.  Not 100% sure, but I don’t think you can do this with RDS.
        5. CloudWatch metrics/graphing is a very bad experience, and want/need better operational insights to what it provides.  Very glad we don’t depend on CW for this.
      2. All of these reasons have come at considerable cost to us as well, of course.
        1. Besides the obvious host management cycles, we have to manage :
          1. MySQL configurations, that have to map to instance sizes.
          2. Optimization and tuning of the configurations, poor-performance root-cause analysis,
          3. MySQL patches/upgrades.
          4. maybe more of the backup process than we’d like to.
          5. maybe more HA requirements than we’d like to; although I’m glad we have more control over this, per my earlier comment regarding downtime.
          6. maybe more of the storage capacity management than we’d like to.
        2. DBA headcount costs.
          1. We’ve gone through two very expensive and hard-to-find folks on this front, plus costly and often not-helpful, cycle-costing out-sourced DBA expertise.
          2. Currently getting by with a couple of experienced engineers in-house and support from CodeFutures as-needed.
      3. As I’ve seen numerous times in the past, AWS ends up building in features that fill gaps that we’ve either developed solutions for, or worked around.
        1. So if some of the RDS limitations can be worked-around, there’s a good chance that the gaps will be filled by AWS in the future.
        2. But it’s doubtful they’ll support sharding any time soon, there’s too much design and application-layer inter-dependencies involved.  Maybe I’m wrong, that’s just my humble opinion.


BigDoor’s Gamification Tips for Publishers

I love gamification and I hate to see it get a bad reputation. As BigDoor’s marketing specialist, I am fascinated by the conversation surrounding gamification. Gamification is a fascinating space because we have a lot to learn about what works and what doesn’t. At BigDoor, we spend as much time as possible educating publishers on what we think makes gamification a success. So I thought I would start sharing tips related to common questions I see popping up.

Tip #1: Keep your users in mind.

A user that comes to your site because they love your products isn’t going to appreciate a gamification solution that distracts from that. Similarly, a user that is coming to your site for some intrinsic value or status may not care about physical rewards. Take a moment (I recommend more) and think about what your site offers users, gamification can amplify your sites value and increase user loyalty, but it can’t add value where there is none and it definitely won’t succeed if it doesn’t line up with what your users want.

Once you’ve nailed down what people value, ask yourself if gamification can help. Gamification can among other things motivate, increase community and challenge users to engage with content. It can also help overcome the hurdle of registration so many publishers often find themselves facing. If you find that your goals can be met by using gamification, focus on using game mechanics that drive users towards the things they already value or educate them on parts of your site that can get them where they want to go.

Remember, gamification is about adding value to your users. If the average visitor can’t understand the value of the game mechanics you’ve added, something has gone wrong.

If you think you are a publisher that could benefit from gamification on your website, contact our gamification experts at BigDoor to see how we can help!

BigDoor ranks #33 in @StartupCity Top 100 List

New Seattle technology startup resource Startup City partnered with SEOmoz to rank the top 100 Seattle Startups and BigDoor came in at #33. The rankings used 6 categories to rate companies and compiled points in each category to a final score (ours was 65). Categories were: Brand Name Searches, G-News Mentions, Domain Authority, Linking Root Domains, and Social Media followers (Twitter/Facebook). Companies at the top of the list include Zillow, Wetpaint and Cheezburger Network. Rankings will be reviewed and ranked monthly. Thank you Startup City for putting this together! We think it is awesome.

To check out the full list, head over to Startup City’s website here.

.@BigDoor celebrates #geekprideday

In true startup form, we love to embrace our geekiness on a daily basis, but today is special for a number of reasons. Not only are we celebrating the unveiling of our awesome new logo, but it’s also Geek Pride Day! In honor of Geek Pride Day we took our geekiness up a notch (or two). Our employees are all rocking their favorite geeky shirts, we are kicking off the weekend with some awesome Star Wars trivia, Portal themed cocktails, and munching on some delicious cupcakes from The Yellow Leaf Cupcake Company (because cupcakes are the official geek food of BigDoor).

If you didn’t know, Geek Pride Day is celebrated on May 25th  and pays tribute to the first Star Wars movie that was released on May 25, 1977. It is also Towel day, a tribute from Douglas Adams Hitchhiker’s Guide to the Galaxy. Last but not least, Terry Pratchett’s fantasy novel Discworld celebrates May 25th as the day for survivors of the People’s Revolution.

If you are letting your Geek Pride show today tell us about it in the comments or on twitter hashtag: #geekprideday!

Winner! BigDoor Rewards Program Sweepstakes

We the just concluded another month of BigDoor Rewards Program Sweepstakes! Three lucky winners have been randomly selected to receive prizes for participating in our online Rewards Programs.

Before we announce the winners, here is a bit of background on the sweepstakes.

Partner websites that have implemented a BigDoor Rewards Program are able to reward users for their online activity with points, badges, and virtual coins. Users can use coins to redeem actual rewards. While each Rewards Program is unique, websites can choose to include prizes that are won via sweepstakes provided by BigDoor free of charge.

This month, users can use their virtual coins to enter to win a $100 Amazon.com Gift Card, a $50 Best Buy Gift Card, and a $25 Starbucks Gift Card. BigDoor randomly picks a lucky winner and sends them their gift card, no strings attached. The online drawing is conducted on the First business day of the month.

This month’s winners are:

$100 Amazon.com Gift Card Winner – Ellie K.

$50 Best Buy Gift Card Winner – Andy S.

$25 Starbucks Gift Card Winner – Jessica S.

Congratulations and thanks for participating online!

If you have a website and are looking to reward your users with cool rewards, sign up for a free trial of the BigDoor platform today!