Skip to content
This repository was archived by the owner on Feb 3, 2026. It is now read-only.

Commit 6ec908d

Browse files
aimurphyaimurphy
authored
move index to streaming learning (#64)
1 parent 9971e19 commit 6ec908d

9 files changed

Lines changed: 161 additions & 67 deletions

‎modules/ROOT/images/avoid-cloud-lockin.svg‎

Lines changed: 5 additions & 0 deletions
Loading

‎modules/ROOT/images/connect-clients-to-astra-db.svg‎

Lines changed: 9 additions & 0 deletions
Loading

‎modules/ROOT/images/migrating-apps.svg‎

Lines changed: 10 additions & 0 deletions
Loading

‎modules/ROOT/images/using-the-astra-console.svg‎

Lines changed: 10 additions & 0 deletions
Loading

‎modules/ROOT/images/what-is-astra-db.svg‎

Lines changed: 6 additions & 0 deletions
Loading

‎modules/ROOT/images/what-is-astra-streaming.svg‎

Lines changed: 5 additions & 0 deletions
Loading

‎modules/ROOT/images/what-is-datastax-luna.svg‎

Lines changed: 6 additions & 0 deletions
Loading

‎modules/ROOT/pages/index.adoc‎

Lines changed: 109 additions & 66 deletions
Original file line numberDiff line numberDiff line change
@@ -1,91 +1,134 @@
1-
= {company} Streaming Learning
1+
= Data streaming and event stream processing
2+
:page-layout: landing
3+
:data-uri:
24

3-
The learning site is a collection of guides and articles about {company} streaming products. Each product has its own area dedicated to providing specifics to get started, its features, and other details. The learning site helps you get the most out of each product and get to production using best practices.
5+
[.[&>h2]:!hidden]
6+
== {empty}
47

5-
== Processing Data
6-
7-
++++
8-
<div class="landing-row">
8+
[subs="macros,attributes"]
99
++++
10+
<div class="lg:row-span-2 flex flex-col items-start gap-4 rounded border p-4">
1011
11-
[sidebar.landing-card]
12-
.xref:streaming-learning:use-cases-architectures:change-data-capture/index.adoc[Change Data Capture (CDC)]
13-
****
14-
--
15-
Change Data Capture (CDC) is a design pattern used in software development to capture and propagate changes made to data in a system. The CDC pattern is commonly used in real-time data streaming applications to enable near-real-time processing of data changes.
16-
--
17-
****
18-
19-
[sidebar.landing-card]
20-
.xref:streaming-learning:use-cases-architectures:real-time-data-pipeline/index.adoc[Real-time data pipeline]
21-
****
22-
--
23-
A real-time data pipeline is a series of steps that takes data from its original source and moves it to a destination such as a data warehouse or data lake. The pipeline is a set of steps that are executed in a specific order to transform the data and make it available for analysis.
24-
--
25-
****
12+
<div class="rounded bg-level1 p-2">
13+
svg:ROOT:what-is-astra-db.svg[role="mx-auto my-auto w-8 h-8"]
14+
</div>
15+
16+
<div><h3 class="discrete !text-h2 !m-0 [&>a]:text-white">Streaming Learning</h3></div>
17+
18+
Learn about the amazing things you can do with all of our streaming products.
19+
We've included best practices for Apache Pulsar, a full connector reference,
20+
and examples for getting the most out of Astra's CDC feature.
21+
22+
xref:streaming-learning:use-cases-architectures:real-time-data-pipeline/index.adoc[Build a real-time data pipeline]
2623
27-
++++
2824
</div>
29-
++++
3025
31-
== Migrating to {pulsar-reg}
26+
<div class="grid lg:grid-rows-1 lg:grid-cols-2 gap-6 mt-6">
3227
33-
++++
34-
<div class="landing-row">
35-
++++
28+
<div class="flex flex-col items-start gap-4 rounded border p-4">
3629
37-
[sidebar.landing-card]
38-
.xref:streaming-learning:use-cases-architectures:starlight/jms/index.adoc[Starlight for JMS]
39-
****
40-
--
41-
Starlight for JMS allows enterprises to take advantage of the scalability and resiliency of a modern streaming platform to run their existing JMS applications.
30+
<div class="rounded bg-level1 flex p-2">
31+
svg:ROOT:using-the-astra-console.svg[role="mx-auto my-auto w-8 h-8"]
32+
</div>
4233
43-
xref:streaming-learning:use-cases-architectures:starlight/jms/index.adoc[Get started now] | xref:starlight-for-jms:ROOT:index.adoc[Configuring] | https://github.com/datastax/pulsar-jms[Source Code]
44-
--
45-
****
34+
<h3 class="discrete !text-h2 !m-0 [&>a]:text-white">Astra Streaming</h3>
4635
47-
[sidebar.landing-card]
48-
.xref:streaming-learning:use-cases-architectures:starlight/kafka/index.adoc[{kafka-for-astra}]
49-
****
50-
--
51-
{kafka-for-astra} brings native {pulsar} protocol support by introducing a Kafka protocol handler on {pulsar-short} brokers.
36+
Delivered as a fully-managed SaaS with boundless scale, massive throughput and low latency,
37+
Astra Streaming is the simplest way to modernize your event driven architecture and turbo charge your data in motion strategy.
5238
53-
xref:streaming-learning:use-cases-architectures:starlight/kafka/index.adoc[Get started now] | xref:starlight-for-kafka:ROOT:index.adoc[Configuring] | https://github.com/datastax/starlight-for-kafka[Source Code]
54-
--
55-
****
39+
xref:astra-streaming:getting-started:index.adoc[Astra Streaming quickstat]
5640
57-
[sidebar.landing-card]
58-
.xref:streaming-learning:use-cases-architectures:starlight/rabbitmq/index.adoc[{starlight-rabbitmq}]
59-
****
60-
--
61-
{starlight-rabbitmq} combines the AMQP 0.9.1 API with {pulsar-short}, providing a powerful way to modernize your RabbitMQ infrastructure, improve performance, and reduce costs.
41+
</div>
6242
63-
xref:streaming-learning:use-cases-architectures:starlight/rabbitmq/index.adoc[Get started now] | xref:starlight-for-rabbitmq:ROOT:index.adoc[Configuring] | https://github.com/datastax/starlight-for-rabbitmq[Source Code]
64-
--
65-
****
43+
<div class="flex flex-col items-start gap-4 rounded border p-4">
44+
45+
<div class="rounded bg-level1 flex p-2">
46+
svg:ROOT:what-is-datastax-luna.svg[role="mx-auto my-auto w-8 h-8"]
47+
</div>
48+
49+
<h3 class="discrete !text-h2 !m-0 [&>a]:text-white">IBM Elite Support for Apache Pulsar</h3>
50+
51+
A production-ready distribution of Apache Pulsar with 24/7 expert enterprise support.
52+
53+
https://www.ibm.com/docs/en/supportforpulsar[Learn more]
54+
55+
</div>
6656
67-
++++
6857
</div>
69-
++++
7058
71-
== APIs & References
59+
<div class="lg:row-span-2 flex flex-col items-start gap-4 rounded border p-4 mt-6">
60+
61+
<div class="rounded bg-level1 p-2">
62+
svg:ROOT:avoid-cloud-lockin.svg[role="mx-auto my-auto w-8 h-8"]
63+
</div>
64+
65+
<div><h3 class="discrete !text-h2 !m-0 [&>a]:text-white">KAAP Operator</h3></div>
7266
73-
=== Connectors
67+
The Kubernetes Autoscaler for Apache Pulsar (KAAP) simplifies
68+
running Apache Pulsar on Kubernetes by applying the familiar operator pattern
69+
to Pulsar's components, and horizonally scaling resources up or down based on
70+
CPU and memory workloads.
7471
75-
A connector is a function that moves data between {pulsar} and external systems. Source are used to push data to {pulsar-short} from external systems such as databases, message queues, and storage systems. Sinks are used to pull data from a {pulsar-short} topic to an external system like a database, data warehouse, or storage system. +
76-
xref:streaming-learning:pulsar-io:connectors/index.adoc#_source_connectors[Sources] | xref:streaming-learning:pulsar-io:connectors/index.adoc#_sink_connectors[Sinks] | xref:streaming-learning:pulsar-io:connectors/index.adoc#_experimental_connectors[Experimental]
72+
xref:kaap-operator::index.adoc[Learn more about KAAP]
7773
78-
=== {pulsar-short} Functions
74+
</div>
75+
76+
<div class="grid lg:grid-rows-1 lg:grid-cols-3 gap-6 mt-6">
77+
78+
<div class="flex flex-col items-start gap-4 rounded border p-4">
79+
80+
<div class="rounded bg-level1 p-2">
81+
svg:ROOT:connect-clients-to-astra-db.svg[role="mx-auto my-auto w-8 h-8"]
82+
</div>
83+
84+
<div><h3 class="discrete !text-h2 !m-0 [&>a]:text-white">Sink and source connectors</h3></div>
85+
86+
Pulsar supports many popular streaming connectors, including:
87+
88+
<ul class="!m-0 [&>li]:my-2">
89+
<li>xref:streaming-learning:pulsar-io:connectors/sinks/elastic-search.adoc[]</li>
90+
<li>xref:streaming-learning:pulsar-io:connectors/sinks/kafka.adoc[]</li>
91+
<li>xref:streaming-learning:pulsar-io:connectors/sinks/snowflake.adoc[]</li>
92+
<li>xref:streaming-learning:pulsar-io:connectors/index.adoc[And more]</li>
93+
</ul>
94+
95+
</div>
96+
97+
<div class="flex flex-col items-start gap-4 rounded border p-4">
7998
80-
Functions are lightweight compute processes that enable you to process each message received on a topic. You can apply custom logic to that message, transforming or enriching it, and then output it to a different topic. +
81-
xref:streaming-learning:functions:astream-functions.adoc[Learn more]
99+
<div class="rounded bg-level1 p-2">
100+
svg:ROOT:migrating-apps.svg[role="mx-auto my-auto w-8 h-8"]
101+
</div>
82102
83-
=== Transformation Functions
103+
<div><h3 class="discrete !text-h2 !m-0 [&>a]:text-white">Starlight API integrations</h3></div>
84104
85-
Transform functions are a low-code implementation of common {pulsar-short} functions. They are used to transform messages from one format to another. Use them to transform a message, enrich messages with additional data, or filter messages based on their content. +
86-
xref:streaming-learning:functions:cast.adoc[Cast] | xref:streaming-learning:functions:compute.adoc[Compute] | xref:streaming-learning:functions:drop.adoc[Drop] | xref:streaming-learning:functions:flatten.adoc[Flatten] | xref:streaming-learning:functions:merge-key-value.adoc[Merge] | xref:streaming-learning:functions:unwrap-key-value.adoc[Unwrap]
105+
Learn about the Starlight API suite to painlessly integrate existing messaging workloads.
87106
88-
=== Topic Subscriptions
107+
<ul class="!m-0 [&>li]:my-2">
108+
<li>xref:starlight-for-jms::index.adoc[Starlight for JMS]</li>
109+
<li>xref:starlight-for-kafka::index.adoc[]</li>
110+
<li>xref:starlight-for-rabbitmq::index.adoc[]</li>
111+
</ul>
112+
113+
</div>
114+
115+
<div class="flex flex-col items-start gap-4 rounded border p-4">
116+
117+
<div class="rounded bg-level1 p-2">
118+
svg:ROOT:what-is-astra-streaming.svg[role="mx-auto my-auto w-8 h-8"]
119+
</div>
120+
121+
<div><h3 class="discrete !text-h2 !m-0 [&>a]:text-white">xref:astra-streaming:developing:astream-cdc.adoc[Change Data Capture]</h3></div>
122+
123+
Change Data Capture (CDC) for Astra Streaming enables you to send data
124+
changes in real time throughout your entire ecosystem. With a wide range of
125+
connectors to data warehouses, messaging systems, data lakes as well as client
126+
libraries, you can send your data wherever it needs to go in real time.
127+
128+
xref:astra-streaming:developing:astream-cdc.adoc[]
129+
130+
</div>
131+
132+
</div>
89133
90-
Subscriptions in {pulsar-short} describe which consumers are consuming data from a topic and how they want to consume that data. +
91-
xref:streaming-learning:subscriptions:astream-subscriptions-exclusive.adoc[Exclusive] | xref:streaming-learning:subscriptions:astream-subscriptions-shared.adoc[Shared] | xref:streaming-learning:subscriptions:astream-subscriptions-failover.adoc[Failover] | xref:streaming-learning:subscriptions:astream-subscriptions-keyshared.adoc[Key Shared]
134+
++++

‎modules/functions/pages/deploy-in-sink.adoc‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
= Deploy transform function in sink
22

3-
As of https://www.ibm.com/docs/en/supportforpulsar[IBM Elite Support for Apache Pulsar (formerly Luna Streaming)] version 2.10.1.6, transform functions can be deployed inside of a sink process. +
3+
With modern Pulsar versions, transform functions can be deployed inside of a sink process. +
44
Before this update, functions transformed data either after it was written to a topic by a source connector, or before it was read from a topic by a sink connector. +
55
This required either an intermediate topic, with additional storage, IO, and latency, or a custom connector. +
66
Now, functions can be deployed at sink creation and apply preprocessing to sink topic writes. +

0 commit comments

Comments
 (0)