OPEA e il Futuro delle Soluzioni GenAI per le Aziende
In un mondo in cui l’Intelligenza Artificiale Generativa continua a ridefinire i confini del possibile, OPEA rappresenta una community di supporto essenziale per tutte quelle organizzazioni che vogliono rimanere al passo con le innovazioni, ma in un modo sicuro e strutturato. Attraverso le sue linee guida, gli eventi di formazione e l’accesso a strumenti open source avanzati, OPEA fornisce alle imprese i mezzi per costruire infrastrutture robuste e scalabili, facilitando al contempo l’integrazione di soluzioni AI direttamente nei loro flussi di lavoro.
Il framework Retrieval-Augmented Generation (RAG) è un esempio perfetto di come un’architettura ben progettata possa trasformare grandi quantità di dati aziendali in insight azionabili, risolvendo uno dei principali ostacoli che molte organizzazioni devono affrontare nel campo dell’AI: il mantenimento dell’accuratezza e della rilevanza delle informazioni generate. Con RAG, non solo le aziende possono migliorare la precisione delle risposte, ma possono anche adattare il sistema alle proprie esigenze specifiche e ai propri dati, creando una soluzione altamente personalizzata.
In un certo senso, OPEA agisce come un acceleratore dell’innovazione aziendale, dove ogni componente – dai modelli di embedding fino alla gestione dei feedback degli utenti – contribuisce a realizzare un sistema AI che cresce e si affina nel tempo. Grazie alla cultura open source, che caratterizza l’approccio di OPEA, le aziende possono anche trarre ispirazione e innovare più rapidamente, incorporando le ultime scoperte e best practice del settore senza la necessità di investire in licenze costose.
Perché Scegliere una Soluzione Open Source?
Optare per un’architettura basata su strumenti open source, come quella proposta da OPEA, offre diversi vantaggi strategici per le aziende:
Flessibilità: Le soluzioni open source possono essere facilmente adattate e integrate con infrastrutture già esistenti, rendendo più agevole il processo di implementazione.
Riduzione dei costi: L’uso di software open source elimina la necessità di pagare per licenze proprietarie, riducendo così i costi operativi e favorendo una gestione più economica dell’innovazione.
Comunità e supporto: Le tecnologie open source sono supportate da una vasta community di sviluppatori, garantendo aggiornamenti costanti e accesso a una rete di esperti pronta ad aiutare.
Innovazione continua: L’open source offre una base collaborativa su cui le aziende possono costruire, sperimentare e condividere idee innovative, spingendo sempre più in là i limiti di ciò che l’AI può fare.
Conclusione
L’approccio di OPEA, unito al potere dell’architettura RAG, offre alle aziende un potente strumento per integrare l’Intelligenza Artificiale Generativa nei loro processi in modo sicuro, scalabile e personalizzabile. In un periodo storico in cui la gestione dei dati e l’automazione sono fondamentali per mantenere un vantaggio competitivo, OPEA fornisce alle organizzazioni tutto ciò di cui hanno bisogno per rimanere al passo con le sfide del mercato.
L’architettura RAG, in particolare, non solo dimostra il potenziale dell’AI nell’estrazione e generazione di insight rilevanti, ma offre anche un metodo sicuro per gestire dati sensibili, grazie all’integrazione di guardrail e sistemi di feedback continui. In questo modo, le aziende possono essere sicure che ogni output generato dal sistema sia accurato e in linea con le esigenze del business.
Scegliere OPEA e un’architettura open source significa abbracciare l’innovazione in modo sostenibile, rimanendo sempre pronti ad adattarsi alle novità del settore. Con OPEA, le organizzazioni hanno il partner ideale per trasformare la promessa dell’Intelligenza Artificiale Generativa in valore concreto e duraturo per il business.
Kubeflow is to MLOps as Kubernetes is to DevOps. Learn how to install it locally to test it out and see if it’s right for you.
MLOps, like its older sibling, DevOps, is a hard problem. And what do engineers do, when faced with hard problems? They create tools to simplify those problems into manageable chunks.
Typical DevOps problems were simplified by using containers, such as Docker. As projects used more and more containers, a container orchestrator, like Kubernetes (K8s) became a necessity. As each problem arose, developers and engineers created amazing solutions to simplify DevOps.
MLOps is going through similar growing pains that DevOps did. You can use Docker and K8s for MLOps, but these get you only so far. MLOps is like a specialized form of DevOps and as such, potentially has different needs or problems. That’s where Kubeflow comes in.
Kubeflow : MLOps :: Kubernetes : DevOps
What is Kubeflow?
Kubeflow is an open-source platform designed to simplifies the process of building and deploying machine learning products at scale on Kubernetes. In fact, it builds on top of K8s, as you can see in the following diagram of it’s place in the stack:
Kubeflow provides a set of tools and frameworks to help with distributed training, serving, and monitoring models.
What can I do with Kubeflow?
Kubeflow can be used by data scientists during the research and experimentation phase. It supports running Jupyter notebooks directly in your K8s cluster, giving you the same hardware as your final deployments.
Data scientists can use Kubeflow to train machine learning models using popular deep learning frameworks like TensorFlow and PyTorch. Kubeflow provides tools for distributed training, hyperparameter tuning, and managing experiments.
Kubeflow can also be used by machine learning engineers to deploy systems for development, testing, and production. It gives them the tools to monitor model performance metrics, so they can react quickly when something changes.
How can I install Kubeflow?
That is the question, isn’t it?
It’s hard to find information on installing Kubeflow locally online. Most documentation online tells you how to install Kubeflow on a cloud services, such as AWS or GCP. This makes sense since, Kubernetes will generally be run on a cluster of computers. However, how can you easily test out Kubeflow to see if it’s right for your situation, if you have to spin up a K8s cluster in the cloud?
Ideally, you would be able to install Kubeflow locally for experimentation and potentially debugging purposes. That’s what this guide is here to show you!
What do I need in order to install Kubeflow locally?
To install Kubeflow locally, you need a few tools.
We’ll assume you have knowledge of Docker, as it is basically a prerequisite for working with Kubernetes. If you are unfamiliar with Docker, check out this tutorial to get you up to speed, then come back here.
Since Kubeflow runs on Kubernetes, you need to be able to run a K8s cluster locally. For that, we turn to minikube. Minikube is a great tool that allows you to quickly setup a local cluster on a single machine. It can run on Linux, macOS, and Windows.
Additionally, you’ll need the following tools:
kubectl, which is a command-line tool to manage your K8s cluster
Using both kustomize and kubectl, you’ll be able to install and setup Kubeflow itself, including a convenient dashboard to control Kubeflow.
Let’s get started!
What is minikube and how do I install it?
As mentioned, minikube is a tool that can help setup a cluster locally on your machine. For a bit of terminology, your computer is a single node, which will host the pods. These pods run your application containers. Pods are manages by a deployment, which describes the desired state of your Kubernetes application.
To install minikube on Linux, check out their handy start page. Here you select your OS, CPU architecture, release type, and installation type. For instance, installing on a Linux machine running an 64-bit x86 processor, you could install minikube with two commands:
The first command downloads the appropriate binary, while the second installs it to the proper location.
What is kubectl and how do I install it?
kubectl is a command line tool to control your K8s cluster. It’s a self-described “swiss army knife”, which “can do many things”. It’s a key tool for managing a cluster.
You have several options to install kubectl. Checkout the installation page for instructions for your platform.
If you use Snaps to manage your app packages, you can install kubectl with a single line:
sudo snap install kubectl --classic
But wanna hear a secret? You don’t need to do any of the above instructions to install kubectl. Why? Because minikube can download the appropriate version of kubectl automatically for you.
If you go this route, the commands will change slightly. For instance, if you want to list (get) all pods running in all namespaces, you would normally run:
kubectl get pods -A
With minikube the command would look like this:
minikube kubectl -- get pods -A
However, to make life easier, you can alias the command:
alias kubectl="minikube kubectl --"
This allows you to use kubectl as you normally would.
What is kustomize and how do I install it?
Kubernetes is controlled by a mountain of YAML files. kustomize is a tool to make customization easier. It lets you customize raw, template-free YAML files, while leaving the original YAML untouched and usable as is.
Check out the installation guide to find an appropriate method to install for your system. For example, to install on Linux you could run:
Build and apply the YAML files for all Kubeflow components:
while ! kustomize build example | awk '!/well-defined/' | kubectl apply -f -; do echo "Retrying to apply resources"; sleep 10; done
Wait for everything to settle out.
You can check to see if everything has settled out by running:
kubectl get pods -A
This will list all pods across all namespaces. As long as the status is Running for all of them, you’re good to go. If any of them are not, sometimes you just need to wait a little longer. Your list should look something like this:
Note: On an Apple Silicon based Mac, Kubeflow may not completely work. At the time of writing there are a couple of critical Pods, whose containers are not available for ARM-based processors. One potential solution would be to run these steps from a Rosetta 2-based Terminal window. For more information check out this StackOverflow answer.
How do I setup a Kubeflow Dashboard?
Once Kubeflow is up and running, you need to do at least one more step, to access the builtin dashboard. You need to forward the correct port to the proper service. The dashboard is accessed via http requests routed through the istio-ingressgateway service in the istio-system namespace. To forward the port, you use kubectl:
This tells your cluster to listen on port 8080 locally and forward it to the service on port 80. You can then reach the dashboard at http://localhost:8080.
The default username is [email protected] and the password is 12341234. Once you log in, you will be greeted by the Kubeflow dashboard.
You can stop everything you’re running by stopping minikube:
minikube stop
But keep in mind that this doesn’t remove your Kubeflow cluster. It just stops it. The next time you run minikube start, it would attempt to restart all your Kubeflow pods and services.
If you want to delete your Kubeflow cluster, run:
minikube delete
Troubleshooting
A few pods are stuck with a STATUS of CrashLoopBackoff
First, wait a little longer. Sometimes it takes quite a few minutes for everything to settle down. If you’ve waited longer than you car, the run:
Are you struggling to understand the differences between load balancers, reverse proxies, and API gateways? Unsure which component is best suited for your web application? You’re not alone! These key components play vital roles in modern web architectures, and knowing how they work is essential for building efficient, secure, and scalable web applications. In this blog, we’ll demystify these concepts and help you make informed decisions for your next project.
Imagine a busy restaurant: the load balancer acts as the headwaiter, ensuring diners are seated evenly to optimize service; the reverse proxy is the skilled waiter who streamlines orders and ensures the best possible experience for each guest; and the API gateway is the restaurant manager who oversees and controls every aspect of the dining experience. Now, let’s translate this analogy into the world of web applications.
Did you know that websites like Amazon and Netflix handle billions of requests every day? With such massive traffic, distributing the load effectively is crucial. Enter the load balancer, which optimizes performance and ensures high availability. But how do load balancers differ from reverse proxies? And when should you use each component? We’ll dive deep into these questions and more.
Lastly, we’ll explore the powerful API gateway, a crucial component in microservices architecture. It centralizes the management, security, and scalability of APIs, ensuring seamless interaction between numerous microservices. But how does the API gateway compare to load balancers and reverse proxies? Stay tuned to find out!
Load Balancer
Have you ever wondered how large-scale web applications handle millions of requests without crumbling under the pressure? The secret lies in the efficient use of load balancers, which ensure optimal resource utilization, maximize throughput, and minimize response time. Let’s dive into the world of load balancing and discover how it can transform your web application.
Did you know that load balancers can operate at different layers of the OSI model? Layer 4 load balancers work at the transport layer, making decisions based on network-level information such as IP addresses and TCP/UDP ports. On the other hand, Layer 7 load balancers operate at the application layer, leveraging HTTP headers, cookies, and URL paths to make more granular decisions. Which approach is best for your application? It depends on your specific needs and requirements, but understanding the distinction is crucial.
b. Load balancing algorithms
So, how do load balancers distribute traffic? They use various algorithms to achieve the best possible distribution. For example, the Round Robin algorithm cycles through available servers, while the Least Connections algorithm directs traffic to the server with the fewest active connections. The Consistent Hashing algorithm, used by many content delivery networks, maps requests to servers based on the request’s hash value, ensuring that similar requests are served by the same server. The choice of algorithm can significantly impact your application’s performance and scalability.
c. Benefits
The benefits of load balancing are immense. For instance, consider the e-commerce giant Amazon, which handles over 2 million transactions per hour. Load balancers play a pivotal role in ensuring high availability and fault tolerance, distributing the workload evenly and rerouting traffic in case of server failures. This keeps your application running smoothly, even during peak traffic times or server outages.
But what about security? Load balancers can also help protect your application from distributed denial-of-service (DDoS) attacks. By distributing traffic evenly and monitoring for unusual patterns, load balancers can detect and mitigate potential threats, safeguarding your application from downtime and performance issues.
d. Summary
In summary, load balancers are essential components for building efficient, secure, and scalable web applications. By understanding their operation, algorithms, and benefits, you can make informed decisions and harness the full potential of load balancing in your projects. Don’t let your application crumble under the pressure of high traffic — embrace load balancing and watch your web application soar to new heights!
Are you looking to enhance the security, performance, and maintainability of your web application? Look no further than the reverse proxy, a versatile and powerful component that can streamline your application and provide a myriad of benefits. Let’s explore how reverse proxies work and why they’re essential for modern web architectures.
a. Operation at Layer 7 (application layer) of the OSI model
Did you know that reverse proxies operate at Layer 7 (application layer) of the OSI model? This means they handle requests and responses at the HTTP level, enabling them to offer advanced features and functionality. For example, reverse proxies can perform URL rewriting, which simplifies complex URLs and improves SEO. But what other benefits can reverse proxies bring to the table?
b. Functionality and Benefits
Imagine offloading resource-intensive tasks, such as SSL termination and content compression, from your backend servers. Reverse proxies can handle these tasks, freeing up your servers to focus on their core responsibilities. The result? Improved performance and reduced server load.
Security is a paramount concern for any web application. Reverse proxies can act as a shield, filtering incoming requests and protecting your backend servers from malicious traffic. By implementing access controls and monitoring requests for suspicious activity, reverse proxies can fortify your application’s security and safeguard sensitive data.
But did you know that reverse proxies can also improve the maintainability of your application? By providing a single point of access, reverse proxies simplify server management and make it easier to scale your application. As your application grows, you can add or remove backend servers without affecting the client-facing interface, streamlining the process of scaling up or down.
Consider the case of GitHub, a popular platform for software development. With millions of users and countless repositories, GitHub relies on reverse proxies to ensure efficient operation and maintain high performance. By offloading tasks like SSL termination and caching, reverse proxies play a crucial role in delivering a seamless user experience.
c. Summary
In conclusion, reverse proxies are indispensable components for building secure, high-performance, and maintainable web applications. By understanding their operation and the benefits they offer, you can harness the full potential of reverse proxies and take your web application to new heights. So, why wait? Embrace the power of reverse proxies and transform your application today!
API Gateway
Are you building a microservices-based application and searching for an efficient way to manage and secure your APIs? The API gateway is the ultimate solution for centralizing management, enhancing security, and ensuring scalability. Join us as we explore the power of API gateways and learn how they can revolutionize your application.
What exactly is an API gateway? In a nutshell, it’s a server that acts as a single entry point for all API calls from clients to your backend services. It’s like the conductor of an orchestra, coordinating all the individual instruments to create a harmonious performance. But what features can API gateways offer to elevate your application?
API Gateway
a. Features and Benefits
Authentication is crucial for securing your APIs, and API gateways excel at this task. By implementing centralized authentication and authorization, API gateways eliminate the need to duplicate security measures across multiple services, simplifying security management and reducing the risk of vulnerabilities.
Have you ever struggled with managing the rate at which clients access your APIs? API gateways can enforce rate limiting, ensuring that your backend services remain stable and performant even during periods of high demand. This helps prevent system overload and ensures a consistent user experience.
How about transforming requests and responses on-the-fly? API gateways can modify requests and responses as needed, allowing you to modify API payloads, convert between data formats, or add custom headers. This adds flexibility and adaptability to your application, making it easier to evolve over time.
Monitoring and logging are essential for maintaining a healthy application, and API gateways have you covered. By providing centralized monitoring and logging capabilities, API gateways make it easy to track performance, diagnose issues, and identify trends across your entire application.
Take the case of Netflix, a global streaming giant with over 200 million subscribers. Netflix relies on API gateways to manage its extensive microservices architecture, providing a scalable and secure solution for handling millions of API calls every day.
Summary
In summary, API gateways are powerful tools for managing, securing, and scaling your APIs in a microservices architecture. By understanding their features and benefits, you can harness the full potential of API gateways and build applications that are both robust and flexible. So, what are you waiting for? Embrace the power of API gateways and elevate your application to new heights!
Comparing Load Balancer, Reverse Proxy, and API Gateway
Do you find yourself confused when trying to differentiate between load balancers, reverse proxies, and API gateways? You’re not alone! These components share some similarities but serve distinct purposes in modern web architectures. Let’s delve into their key differences and learn how to choose the right component for your application.
a. Key differences in purpose, operation, and features
How do load balancers differ from reverse proxies? While both components distribute requests, load balancers primarily focus on improving performance, availability, and fault tolerance by distributing traffic among multiple backend servers. Reverse proxies, on the other hand, operate at the application layer and offer additional functionality such as URL rewriting, content compression, and access control. So, when should you choose a load balancer or a reverse proxy? It depends on your specific requirements and whether you need advanced application-level features.
But what about API gateways? How do they fit into the picture? API gateways are best suited for microservices architectures, where multiple APIs need centralized management, security, and scalability. Unlike load balancers and reverse proxies, API gateways offer advanced features like authentication, rate limiting, request/response transformation, and monitoring. If your application relies heavily on APIs, an API gateway is an invaluable addition to your architecture.
API Gateway vs. Load Balancer
b. Combining components for optimal web architectures
Can you combine these components for optimal results? Absolutely! In many cases, you’ll find load balancers, reverse proxies, and API gateways working in harmony to create efficient, secure, and scalable web applications. For example, you could use a load balancer to distribute traffic among multiple reverse proxies, which in turn secure and optimize requests to your backend services. Alternatively, you might use an API gateway in conjunction with a load balancer to manage and scale your APIs while maintaining high availability.
Consider the case of Spotify, a leading music streaming platform. Spotify employs a combination of load balancers, reverse proxies, and API gateways to handle millions of daily requests, ensuring a seamless and enjoyable experience for its users.
c. Summary
In conclusion, understanding the unique aspects of load balancers, reverse proxies, and API gateways is crucial for building modern web applications. By comparing their purposes, features, and use cases, you can make informed decisions and choose the right components for your application. Don’t let confusion hold you back — embrace these powerful tools and take your web architecture to the next level!
Popular Tools and Solutions
Are you eager to dive into the world of load balancers, reverse proxies, and API gateways but unsure which tools to use? Fear not! Here is a list of popular and powerful solutions that can help you build and optimize your web application. Let’s explore these industry-leading tools and discover the perfect fit for your needs.
a. Load balancers: HAProxy, NGINX, Amazon ELB
When it comes to load balancers, you can’t go wrong with HAProxy, an open-source, high-performance TCP/HTTP load balancer. Trusted by tech giants like GitHub and Stack Overflow, HAProxy provides a wide range of features, including Layer 4 and Layer 7 load balancing, SSL termination, and health checks. Looking for a cloud-based solution? Check out AWS Elastic Load Balancing, which offers application and network load balancing for Amazon Web Services applications.
b. Reverse proxies: NGINX, Apache HTTP Server, Microsoft IIS
What about reverse proxies? One standout option is NGINX, an open-source web server, reverse proxy, and load balancer all rolled into one. With its high performance and extensive feature set, NGINX is an ideal choice for securing and optimizing your web application. Alternatively, you might consider Apache HTTP Server with its mod_proxy module, which provides reverse proxy functionality for the widely used web server.
c. API gateways: Kong, Amazon API Gateway, Apigee
In the realm of API gateways, look no further than Kong, an open-source, high-performance API gateway built on top of NGINX. With its extensive plugin ecosystem and support for custom plugins, Kong allows you to tailor your API gateway to your application’s unique requirements. If you prefer a fully managed, cloud-based solution, consider Amazon API Gateway, which provides a serverless API management platform with robust security, monitoring, and scaling capabilities.
Curious about container orchestration platforms? Kubernetes, the popular open-source container orchestration platform, offers built-in support for load balancing, reverse proxies, and API gateways through its Ingress and Service resources, making it an excellent choice for containerized applications.
d. Summary
In conclusion, the world of load balancers, reverse proxies, and API gateways is filled with powerful and flexible tools to suit every need. By exploring popular solutions like HAProxy, NGINX, Kong, and Kubernetes, you can make informed decisions and find the perfect tools for your web application. Don’t let uncertainty hold you back — embrace these cutting-edge solutions and elevate your web architecture today!
Conclusion
In conclusion, navigating the intricate world of load balancers, reverse proxies, and API gateways can be a daunting task, but armed with the right knowledge, you’re now equipped to make informed decisions and choose the perfect components for your web application. Remember that load balancers distribute traffic among multiple backend servers, reverse proxies offer additional application-level features, and API gateways provide centralized management and security for microservices-based applications.
Next, don’t hesitate to mix and match these components for optimal results. By combining load balancers, reverse proxies, and API gateways, you can build a web architecture that is efficient, secure, and scalable. Embrace the power of these components and unlock the full potential of your application.
Furthermore, explore the wide array of powerful tools and solutions available, such as HAProxy, NGINX, Kong, and Kubernetes. Each tool offers unique benefits, features, and use cases, allowing you to find the perfect fit for your application. Leverage these industry-leading tools and stay ahead of the curve.
Finally, remember that the technology landscape is ever-evolving, and staying up-to-date with the latest advancements, trends, and best practices is crucial for building cutting-edge web applications. Continuously learn, experiment, and innovate to ensure your application remains competitive and future-proof.
As you embark on your journey to build a robust, high-performance web application, take these lessons to heart and apply them in your projects. By understanding the power of load balancers, reverse proxies, and API gateways, and leveraging the best tools and practices, you can elevate your web architecture and create extraordinary experiences for your users. So, what are you waiting for? Embrace the world of load balancers, reverse proxies, and API gateways, and transform your application today!
Applications grow. As an application attracts more users, so do the databases that store the information created, whether that’s sales transactions or scientific data gathered. Larger datasets require more resources to store and process that data. Plus, with more simultaneous users using the system, the database needs more resources.
When your application becomes popular, it needs to scale to meet the demand. Nobody sticks around if an application is slow — not willingly, anyway.
If you need to scale, celebrate it as a good problem! But that doesn’t make the process simple. Scaling has multiple possible options, each requiring different levels of sophistication. Here, we cover scaling both as a generic challenge and specifically for Redis databases, with attention to advanced scaling using Redis Enterprise.
Scaling Concepts
Scaling is a multidimensional problem with several distinct solutions.
Vertical scaling involves increasing a database’s resources. Typically, this involves moving the database to a more powerful computer or to a larger instance type.
“More” is the key word. As with any hardware choice, you consider more powerful processors, more memory and/or more network bandwidth. You have to find a balance between them that optimally improves the database’s performance and the number of simultaneous users it can support, not to mention optimizing your hardware budget.
One element in vertical scaling is adjusting the amount of RAM available to the database. In the case of Redis, RAM limits the amount of data that the database can store, so it’s an important consideration.
Vertical scaling is colloquially called scaling up or scaling down, depending on whether you move up to a more powerful computer or (in rare circumstances) shift down to a less powerful computer.
Redis provides a competitive edge to any business by delivering open source and enterprise-grade data platforms to power applications that drive real-time experiences at any scale. Developers rely on Redis to build performance, scalability, reliability, and security into their applications.
Horizontal scaling involves adding additional computer nodes to a cluster of instances that operate the database, without changing the size or capacity of any individual node. Horizontal scaling is also called scaling out (when you add nodes) or scaling in (when you decrease the number of nodes).
Depending on how it’s implemented, horizontal scaling can also improve the database’s overall reliability. It eliminates a single point of failure because you are increasing the number of nodes that can be used in failover situations. However, horizontal scaling also increases time and effort (and thus costs), because you need more nodes (and hence more failure points) to keep the database functional.
In other words, vertical scaling increases the size and computing power of a single instance or node, while horizontal scaling changes the number of nodes or instances.
Vertical scaling is an easy way to improve database performance, assuming that you have or can acquire a larger computer or instance. It typically can be implemented easily in the cloud, with no impact on the application or database architecture.
Complexity isn’t a bad thing when it’s the right choice, as long as you know what you are doing.
When done correctly, horizontal scaling gives your database and your application significantly more room to grow. This scheme has plenty of history in responding to performance bottlenecks: Just throw more hardware at it!
However, horizontal scaling typically is harder to implement than vertical scaling. Adding additional nodes means more complexity. Are those nodes read-only nodes? Read/write master nodes? Active masters? Passive masters? The complexity of your database and your application architecture can increase dramatically.
Complexity isn’t a bad thing when it’s the right choice, though, as long as you know what you are doing.
There are several ways to implement horizontal scaling, each with a distinct set of advantages and disadvantages. Selecting the right model is important in building a data storage architecture. Redis supports many horizontal scaling options. Some are available in Redis open source (OSS), and some are available only in Redis Enterprise.
The Basics of Sharding
Sharding is a technique for improving a database’s overall performance, as well as increasing its storage and resource limits. It’s a relatively simple horizontal scaling technique.
With sharding, data is distributed across various partitions, or nodes. Each node holds only a portion of the data stored in the entire database. In Redis’ case, a key/value input is processed, and the data is stored in a shard.
When a request is made to the database, it is sent to a shard selector, which chooses the appropriate shard to send the request. In Redis, shard selection is often implemented by a proxy that looks at the key for the requested data, and based on the key, the proxy sends the request to the appropriate shard instance.
The shard selection algorithm is deterministic, which means every request for a given key always goes to the same shard. Only that shard has information for a given data key, as illustrated by Figure 1.
Figure 1. Horizontal scaling via sharding
Sharding is a relatively easy way to scale out a database’s capacity. By adding three shards to a Redis OSS implementation, for instance, you can nearly triple the database’s performance and triple the storage limits.
But sharding isn’t always simple. Choosing a shard selector that effectively balances traffic across all nodes may require tuning. Sharding can also lower application availability because it increases dependency on multiple instances. If you don’t manage it properly, failure of a single instance can bring down the entire database. That’ll cause a bad day at work.
Redis clustering addresses these issues, and also makes sharding easier to implement. If resharding is necessary to rebalance a database for reasons of storage capacity or performance, the data is physically moved to a new node.
An application’s awareness of the shard selector algorithm can allow the application to perform better overall balancing across the shards, though at the cost of increased complexity.
Sharding effectiveness is only as good as the shard selector algorithm that is used. An application’s awareness of the shard selector algorithm can allow the application to perform better overall balancing across the shards, though at the cost of increased complexity.
Clustering in Redis OSS is led by the cluster with the client library being cluster-aware. Essentially, the shard selector is implemented in the client library. This requires client-side support of the clustering protocol.
In Redis Enterprise, a server-side proxy is used to implement the shard selector and to provide support for clustering server side. The proxy acts as a load balancer of sorts between the horizontally scaled Redis instances.
Clustering is a common solution to horizontal scaling, but it has pros and cons. On the plus side, sharding is an effective way to quickly scale an application, and it is used in many large, highly scaled applications. Also, it is available out of the box.
On the other hand, clustering requires additional management. You need to know what you’re doing. Individual, large keys can create imbalances that are difficult or impossible to compensate for.
Redis clustering eliminates much of shardings’ complexities. It allows applications to focus on the data management aspects of scaling a large dataset more effectively. It improves both write and read performance.
Ultimately, how well it all works depends on the access patterns that the application uses.
Read Replicas
Another horizontal scaling option is read replicas. As the name suggests, the emphasis of read replicas is to improve the performance of reading data without regard to the time spent writing data to the database. The premise is that it is far more common to retrieve data than to change it or to add new data.
In a simple database, data is stored on a single server, and both read and write access to the data occurs on that server. With read replicas, a copy of the server’s data is stored on auxiliary servers, called read replicas. Whenever data is updated, the replicas receive updates from the primary server.
Each auxiliary server has a complete copy of the database. So when an application makes a read request, that request can go to any of the read replica servers. That means a significantly greater number of read requests can be handled simultaneously, which improves scalability and overall performance.
Read replicas cannot improve write performance, but they can increase read performance significantly.
But read replicas have limitations, depending on several factors, such as the consistency model that the database uses, or the network latency you need to contend with.
Read replicas cannot improve write performance, but they can increase read performance significantly. However, that does require you to consider how write-intensive your application is. It takes some time for a database write to the master database to propagate to the read replicas. This delay, called skew, can result in older data being returned to the application while the primary server updates the replica servers. The delay is only for a short period of time, but sometimes those delays are critical. This may or may not be an issue for your own situation, but take note of the issue as you design your system.
Think about the process of writing to a database.
When you update information or add new data, the write is performed to the master database instance only. That’s sacrosanct; all writes must go to the one master database instance.
This master instance then sends a message to all of the read replicas, indicating what data has changed in the database, and enabling the read replicas to update their copies of the data to match the master copy.
Since all database writes go through the master instance, there is no write performance improvement when additional read replicas are added. In fact, there can be a minor decrease in write performance when you add a new read replica. That’s because the master now has an additional node it must notify when a write occurs. Typically, this impact is not significant, but it’s certainly not a zero impact.
Consider the illustration in Figure 2, which shows a Redis implementation consisting of three servers. All writes to the Redis database are made to the single master database. This single master sends updates of the changed data to all the replicas. Each replica contains a complete copy of the stored Redis database.
Then, when the application wants to retrieve data, the read access to the Redis instance can occur on any server in the cluster. A load balancer takes care of routing the individual read requests, which directs traffic using one of a number of load balancing algorithms. (There are several load balancing algorithms, including round robin, least used, etc., but they are outside the scope of this discussion.)
Figure 2. Horizontal scalability with read replicas
Another benefit to using read replicas is in availability improvement. If a read replica crashes, the load balancer simply redirects traffic to another read replica. If the write master crashes, you can promote one of the read replicas to the role of master, so the system can stay operational.
Read replicas are an easy-to-implement model for horizontal scalability, and the method improves availability with little or no application impact.
Active-Active
Active-Active replication or Active-Active clustering is a way to improve performance for higher database loads.
As with read replicas, Active-Active (also called multimaster replication) relies on a database cluster with multiple nodes, with a copy of the database stored on all the nodes and a load balancer distributing the load.
With Active-Active replication, however, both read and write requests are distributed across multiple servers, and load balanced among all the nodes. The performance boost is meaningful, because a significantly larger number of requests can be handled, and they are handled faster.
Note that Active-Active replication is not supported directly by Redis OSS. If this turns out to be the appropriate scaling architecture for your needs, you’ll need Redis Enterprise. But the focus here is in explaining the computer science technique, no matter where you get it from (including building it yourself, if you have that sort of time).
With Active-Active replication, the read propagation happens exactly as described in the previous section.
When an application writes to one node, this database write is propagated to every master in the system. There are many ways this can occur, such as:
The application can force the write to all masters.
A write proxy can distribute the writes.
The master node that receives the write call can forward the request to other non-receiving master servers.
Figure 3 illustrates a database implementation with a cluster consisting of three servers. Each server contains a complete copy of all the data. Any server can handle any type of data request — read or write — for any data in the database.
Figure 3. Active-Active replication
When the load balancer directs a write request to the database master instance, such as in this example, it sends the update to all the other replicated instances. If a write is sent to any of the other nodes, that node sends the update to all the other replicated instances in a similar manner.
But what happens when two requests are sent to update the same data value?
In a single-node database, the requests are serialized and the changes take place in order, with the last change typically overriding previous changes.
In the Active-Active model, though, the two requests could come to different masters. The masters could then send conflicting update messages to the other master servers. This is called a write conflict.
In the case of a write conflict, the application needs to determine which database-write to keep and which to reject. That requires a resolution algorithm of some type, involving application logic or database rules.
Additionally, since updates are sent to each node asynchronously, it’s possible for data lag to cause one node to go slightly out of sync with another node. That’s an issue even when that mismatch is only for a short period of time. Developers have to take care that the application considers this potential lag so that it does not affect operations. This is similar to the issues with read replicas, but is potentially more complex.
Besides improving performance, this model of horizontal scalability also increases overall database availability. If a single node fails, the other nodes can take up the slack. However, since each node contains a complete copy of the data, this has no impact on the storage limit of a database by adding additional servers.
The cost of this model is increased application complexity in dealing with conflicting data.
Redis Enterprise’s Active-Active Geo-Distribution
Redis OSS does not natively support multimaster redundancy in any form.
However, Redis Enterprise does Active-Active Geo-Distribution, which provides Active-Active multimaster redundancy.
Then Redis Enterprise’s Active-Active Geo-Distribution goes one step further. It enables individual clusters to be located in geographically distributed locations yet replicate data between them. Take a look at Figure 4.
Figure 4. Active-Active replication across geography
This allows a Redis database to be geographically distributed to support software instances running in different geographic locations.
In this model, multiple master database instances are in different data centers. Those can be located across different regions and around the world. Individual consumers, via the application, connect to the Redis database instance that is nearest to their geographic location. The Active-Active Redis database instances are then synchronized in a multimaster model so that each Redis instance always has a complete and up-to-date copy of the data.
Redis Enterprise’s Active-Active Geo-Distribution has sophisticated algorithms for effectively dealing with write conflicts, including implementing conflict-free, replicated data types (CRDTs) that guarantee strong eventual consistency and make the process of replication synchronization significantly more reliable. The application still must be aware of and deal with data lag and write conflicts, so these issues don’t become a problem.
What’s Right for You?
You need to make your applications run faster and support additional burden on their databases. Fortunately, as this article demonstrates, you have plenty of options for scaling techniques. Each has a different impact on the amount of storage space available to your application and on the system resources.
The technique you ultimately choose depends on many factors, including your company’s goals, your software requirements, the skills of the people in your IT department, your application architecture and how much complexity you’re willing to take on.
To learn more about how Redis Enterprise scales databases — with more diagrams, which are always fun — consult “Linear Scaling with Redis Enterprise.”
Vincono perché risolvono la complessità della comunicazione.
La legge di Metcalfe dice che ogni volta che aggiungi un nuovo utente a una rete, il numero di connessioni aumenta proporzionalmente al quadrato del numero di utenti. Questa è tecnologia, M cosa significa questo per la tua organizzazione e i tuoi team?
La struttura della nostra mente è che non può davvero gestire un gran numero di relazioni. Scienziati come Robin Dunbar ci hanno detto per decenni che il nostro mondo sociale è molto piccolo.
Dicono che ci sono circa 5 persone con cui possiamo avere relazioni strette e altre 15 persone con cui possiamo avere relazioni leggermente meno intense. Pensa alle squadre sportive, per esempio. Raramente sono più grandi di 15 persone.
Per questo motivo, le organizzazioni progressiste si sono allontanate dalle tradizionali gerarchie manageriali. Al contrario, si strutturano come reti decentralizzate di team.
Queste reti non hanno (o pochi) quadri intermedi. Sono dotati di team altamente autonomi in cui i membri si occupano della comunicazione, del coordinamento e della contrattazione.
Ma come mostra la legge di Metcalfe, quando non ci sono manager, i team devono essere abbastanza piccoli da non sovraccaricare i membri di comunicazioni e informazioni
