Setting up tables
Data typically resides in tables. When sharing data with Toplyne via S3, the best practice is creating separate tables for event data and partitioning profile data on a time column.
Event data table
This table will capture a user's action; it will help answer the question, “what has a user done on your product, and when.”
Rows
Each row in this table represents an event triggered by a user.
Columns
USER_ID
(Required): Key to identify which user has performed the eventACCOUNT_ID
(required for account-level analytics): Key to identify which account the particular user belongs toEVENT_NAME
(Required): Name of the eventTIMESTAMP
(Required): Timestamp at which the event has occurredUPDATED_AT
(Required): UTC timestamp when the row has been updated or added (not the event timestamp)EVENT_PROPERTIES
(Optional): Event properties, typically shared as a JSON
with key/value pairs
Partition
Partition this table on TIMESTAMP
column.
Sample Event Table
USER_ID | ACCOUNT_ID | EVENT_NAME | TIMESTAMP | UPDATED_AT | EVENT_PROPERTIES |
---|---|---|---|---|---|
1298 | 23 | Payment_Initiated | 2022-10-26 07:08:34 | 2023-01-09 17:08:34 | { "paymentid": 123, "amount" : 200, } |
1567 | 77 | Trial_Ended | 2022-11-01 20:01:14 | 2023-01-09 17:08:34 | { "accountid" : 77 } |
Profile data table
This table will capture any profile information a user or an account has; it will help answer the question, “who is this user or account?”.
Rows
Each row in this table is a unique entity.
Columns
USER_ID
/ACCOUNT_ID
(Required): Key to identify which entities' properties are listed in the row.USER_CREATION_TIME
/ ACCOUNT_CREATION_TIME (Optional): To identify since when the user or account has been active.UPDATED_AT
: UTC timestamp when the row has been updated or added (not the event timestamp)USER_PROPERTIES
/ACCOUNT_PROPERTIES
: Each subsequent column is a profile key; against it, you will have its value. For example, you may choose to include the email address, geographical location, account name, current billing plan they are on, etc.
Partition
Partition this table on UPDATED_AT
column.
Sample User Table
USER_ID | USER_CREATION_TIME | UPDATED_AT | Country | Current Plan | |
---|---|---|---|---|---|
1296 | 2022-10-26 07:08:34 | 2022-11-26 07:08:34 | freddie@queen.com | UK | Annual Plan |
2676 | 2022-11-01 20:01:14 | 2022-12-01 20:01:14 | _john@beetles.com | UK | Monthly Plan |
Sample Account Table
ACCOUNT_ID | ACCOUNT_CREATION_TIME | UPDATED_AT | AccountName | Country | Current Plan |
---|---|---|---|---|---|
23 | 2022-10-26 07:08:34 | 2022-11-26 07:08:34 | Queen | UK | Annual Plan |
56 | 2022-11-01 20:01:14 | 2022-11-26 07:08:34 | Beetles | UK | Monthly Plan |
Step-by-step guide to share
Prerequisites
To connect your AWS S3 bucket to Toplyne, you need:
- An S3 bucket containing files with supported file types and encodings.
- For private or encrypted buckets, an AWS account with the ability to grant Toplyne permission and to read from the bucket.
To authorize Toplyne to connect to your S3 bucket, follow these instructions:
We recommend disabling Access Control Lists (ACLs) on each S3 bucket so that the bucket contents are controlled by the bucket's access control settings and not the original file owner's settings. For more information about disabling ACLs for your bucket, see AWS S3 documentation.
Create a bucket for Toplyne
- We recommend creating a bucket exclusively for toplyne data as it makes it very very easy for you to share data with us.
- Open your S3 console and navigate to the permissions tab.
<https://s3.console.aws.amazon.com/s3/buckets/><bucket_name>?tab=permissions
- We want you to grant toplyne read-only permissions to this bucket.
- Navigate to the Bucket policy and Edit it.
- Paste this JSON and update the bucket policy field -
your-bucket-name
there.
{ "Version": "2012-10-17", "Statement": [ { "Sid": "1", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::388856629077:role/ToplyneServiceRoleForCustomerDataAccess" }, "Action": [ "s3:GetObject", "s3:GetObjectVersion", "s3:ListBucket", "s3:GetBucketLocation" ], "Resource": [ "arn:aws:s3:::{your-bucket-name}", "arn:aws:s3:::{your-bucket-name}/*" ] } ] }
Share your S3 bucket name with the Toplyne team.
Updated almost 2 years ago