Bigtable 分布式计数_AI阅读总结 — 包阅AI

Image provides an example of a counter that you have probably seen before that is tracking engagement as likes, dislikes, and views over time.

Let’s now take a look at how Bigtable can streamline your analytics workflows.

Example use cases

Imagine you’re running a social media app and need to track billions of interactions with content every day throughout the world, such as the total number of impressions, views, and clicks. This use case is not much different than tracking interactions with ads in marketing, product listings in retail or page views on any website. In the past, you had two main options. Either you could write each activity to the database and then run multiple queries to calculate the totals; that can be expensive, result in delayed metrics, and negatively impact serving performance during processing, even on specialized columnar systems. Or you could turn to a streaming framework, which, while capable of handling real-time data, has a complicated architecture with management overhead and significant additional costs.

With Bigtable, you can simply increment a counter for each activity as it occurs, which makes the total count available within Bigtable instantly. You can also decrement, even delete or reset counts, and the numbers will synchronize. The results can be read from a dashboard or application with single-digit millisecond latency, providing your global workforce, partners, advertisers, and customers with up-to-date key engagement metrics for various campaigns and marketing initiatives.

While seemingly simple, counting can be quite powerful. Consider these additional use cases:

• Web analytics: Measure Unique visitors, daily/weekly/monthly active users.

• AdTech: Track impressions, engagements, clicks for ads served.

• IoT & telemetry: Aggregate sensor data, track device usage, or identify trends and anomalies.

• Social media: Measure user interactions, analyze content popularity, or track trending topics.

• Gaming: Monitor player activity, track achievements, or analyze game performance.

• E-commerce: Track sales, inventory levels, or customer engagement metrics in real time.

Benefits

• Efficient data aggregation: Automatically generate metrics in Bigtable on any table. Merge values (+/-) or delete/reset counters. Metrics can be stored across time windows and retrieved with low-latency lookups.

• Massive throughput: Build metrics on streaming data with tens of thousands of mutations per second per Bigtable node even on a single metric.

• Global consistency: Metrics can be consolidated throughout the globe and any inconsistencies are automatically resolved and won’t diverge (i.e. they’re eventually consistent).

Building your first counter

Building a counter in Bigtable is straightforward. For this walkthrough, let’s build on our previous example that is tracking billions of daily user interactions throughout the world. Let’s say we now want to track the unique individuals that are engaging with each post.

First, create an aggregate column family, specifying the desired aggregation function. This may be a SUM, MIN, MAX, or, for our example, the HyperLogLog (HLL) data type, which lets us approximately count by unique values. Bigtable relies on HLL++ sketch for this approximation.

In the code snippets below, we assume we have an existing table called posts where each row is a social media post that’s already storing the text copy, author, etc. We will add a counter directly to this table.

We will use Bigtable’s command line tool (cbt) to add a column family called count that has no garbage collection policy (never) and is of HLL type, that will accept integers as inputs (inthll).