Scaling Shiny Apps for Python and R: Sticky Sessions on Heroku | R-bloggers

• Read original article here

Scaling Shiny Apps for Python and R: Sticky Sessions on Heroku
Posted on November 7, 2022 by Peter Solymos in R bloggers | 0 Comments
[This article was first published on R - Hosting Data Apps , and kindly contributed to R-bloggers ]. (You can report issue about the content on this page here )
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.
Share Tweet
Shiny for R and Python can be deployed in conventional ways, using RStudio Connect, Shiny Server Open Source, and Shinyapps.io. These hosting solutions are designed with scaling options included and are the officially recommend hosting options for Shiny apps.
When it comes to alternative options, the docs tell you to have support for WebSockets and to use sticky load balancing. WebSocket support is quite common, but load balancing with session affinity is not a trivial matter, as illustrated by this quote:
We had high hopes for Heroku, as they have a documented option for session affinity. However, for reasons we don’t yet understand, the test application consistently fails to pass. We’ll update this page as we find out more. – Shiny for Python docs
In this post and the associated video tutorial, we are going to investigate what is happening on Heroku and whether we can make sticky load balancing work.
This tutorial is a written version of the accompanying video :
Prerequisites
You'll need Git and the Heroku CLI . We will implement scaling, which implies that at least the Standard dyno type due to the default scaling limits .
⚠️
Starting November 28, 2022, free Heroku Dynos, free Heroku Postgres, and free Heroku Data for Redis will no longer be available – see this FAQ for details.
Test applications
The Shiny for Python docs p roposes a test to make sure that your deployment has sticky sessions configured. The application sends repeated requests to the server, and the test will only succeed if they connect to the same server process that the page was loaded on.
We build a Python and an R version of a test application to test how load balancing works. We use this Shiny for Python test application .
Shiny for Python
This is how the test application looks in Python (see the load-balancing/app.py file, which is written by Joe Cheng ):
from shiny import * import starlette.responses app_ui = ui.page_fluid( ui.markdown( """ ## Sticky load balancing test - Shiny for Python The purpose of this app is to determine if HTTP requests made by the client are correctly routed back to the same Python process where the session resides. It is only useful for testing deployments that load balance traffic across more than one Python process. If this test fails, it means that sticky load balancing is not working, and certain Shiny functionality (like file upload/download or server-side selectize) are likely to randomly fail. """ ), ui.tags.div( {"class": "card"}, ui.tags.div( {"class": "card-body font-monospace"}, ui.tags.div("Attempts: ", ui.tags.span("0", id="count")), ui.tags.div("Status: ", ui.tags.span(id="status")), ui.output_ui("out"), ), ), ) def server(input: Inputs, output: Outputs, session: Session): @output @render.ui def out(): # Register a dynamic route for the client to try to connect to. # It does nothing, just the 200 status code is all that the client # will care about. url = session.dynamic_route( "test", lambda req: starlette.responses.PlainTextResponse( "OK", headers={"Cache-Control": "no-cache"} ), ) # Send JS code to the client to repeatedly hit the dynamic route. # It will succeed if and only if we reach the correct Python # process. return ui.tags.script( f""" const url = "{url}"; const count_el = document.getElementById("count"); const status_el = document.getElementById("status"); let count = 0; async function check_url() {{ count_el.innerHTML = ++count; try {{ const resp = await fetch(url); if (!resp.ok) {{ status_el.innerHTML = "Failure!"; return; }} else {{ status_el.innerHTML = "In progress"; }} }} catch(e) {{ status_el.innerHTML = "Failure!"; return; }} if (count === 100) {{ status_el.innerHTML = "Test complete"; return; }} setTimeout(check_url, 10); }} check_url(); """ ) app = App(app_ui, server)
The UI is nothing but some text and some placeholders inside a card. The server function does two things: it defines a dynamic endpoint that the client will try to connect to repeatedly. This endpoint will send a 200 (OK) status code.
This URL is also hard-coded into a JavaScript snippet that is sent to the client. This JavaScript function is responsible for making a request to the URL 100 times. The test will succeed when the client hits the same dynamic route 100 times.
Important to not that we set the "Cache-Control" header to "no-cache" because browsers tend to cache responses from the same URL. This header helps to avoid that – caching would defeat the purpose of this test.
When load balancing is introduced, the dynamic URL will be different for each instance. If the load balancing is “sticky”, the same client will not end up on a different server process. However, if this is not the case, the test will fail and we will know.
The test application is also built with Docker so that we can deploy the same app to multiple hosting providers without too much hassle.
Hosting Data AppsPeter Solymos
We use the Dockerfile.lb and the app in the load-balancing folder containing the app.py and requirements.txt files:
FROM python:3.9 RUN addgroup --system app && adduser --system --ingroup app app WORKDIR /home/app RUN chown app:app -R /home/app COPY load-balancing/requirements.txt . RUN pip install --no-cache-dir --upgrade -r requirements.txt COPY load-balancing . USER app EXPOSE 8080 CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8080"]
Build, test run the image, and push to Docker Hub:
# build docker build -f Dockerfile.lb-live -t analythium/python-shiny-lb:0.1 . # run: open http://127.0.0.1:8080 docker run -p 8080:8080 analythium/python-shiny-lb:0.1 # push docker push analythium/python-shiny-lb:0.1
You can either build the image yourself or pull it from Docker Hub with docker pull analythium/python-shiny-lb:0.1.
When you run the container and visit http://127.0.0.1:8080 in your browser you'll see the counter increase while the backend will log the requests:
Shinylive
Shinylive is an experimental feature (Shiny + WebAssembly) for Shiny for Python that allows applications to run entirely in a web browser, without the need for a separate server running Python. You can build the load-balancing test application as a fully static app and containerize it based on the Dockerfile.lb-live file.
The docs folder in the repository contains the exported Shinylive site with the static HTML. The app is also deployed to GitHub Pages . When users load the app from a static site, there is no need to do any kind of load balancing, because after your browser downloads the HTML and other static assets, everything happens on the client side.
Shinylive test app.
Shiny for R
The R version is a port of the Python app (see the load-balancing-r/app.R file):
library(shiny) library(bslib) ui