vnc-forge

The core of a Pro-level Android VNC client — SSH tunneling, the RFB protocol, and gesture-based touch control.

File structure

app/src/main/java/com/boffin/vncforge/
├── SshTunnelManager.kt      # SSH local port forwarding via JSch
├── RfbProtocol.kt           # RFB 3.3/3.7/3.8 handshake + VNC Auth (DES) + Raw/CopyRect/Hextile/ZRLE decoding
├── VncCanvasView.kt         # SurfaceView renderer + pinch-zoom/pan/long-press gestures
├── ClipboardSyncManager.kt  # Two-way clipboard sync
├── KeySyms.kt               # Keysym constants and modifier state for the virtual keyboard
├── VirtualKeyboardOverlay.kt# Floating on-screen keyboard (Esc/Ctrl/Alt/Shift/Tab/arrows/Enter)
├── VncSessionController.kt  # Wires all the components together (orchestration)
├── VncEngine.kt             # JNI bridge (extension point for a future native fast path)
├── SavedConnection.kt       # Saved connection data model + ConnectionType enum
├── ConnectionStore.kt       # SharedPreferences+JSON persistence for saved connections
├── ConnectionListActivity.kt# Home screen — saved connections grid
├── ConnectionAdapter.kt     # RecyclerView adapter for the connection grid
├── ConnectionEditActivity.kt# Connection editor form
└── VncActivity.kt           # The actual VNC session screen

app/src/main/java/com/boffin/vncforge/decoder/
├── VncDecoder.kt             # Common interface (IntArray-based, decoupled from Bitmap)
├── PixelUtils.kt             # Shared byte<->ARGB conversion logic (used by Raw/Hextile/ZRLE)
├── HextileDecoder.kt         # Encoding 5 — Raw/BackgroundSpecified/ForegroundSpecified/AnySubrects/SubrectsColoured
└── ZrleDecoder.kt            # Encoding 16 — persistent Inflater, 64x64 tiles, Solid/Packed Palette/Plain RLE/Palette RLE

app/src/main/cpp/
├── CMakeLists.txt
└── vnc_engine.cpp            # JNI stub (currently a no-op placeholder)

.github/workflows/
└── build-apk.yml             # CI: builds debug + release APKs on every push

Screen architecture (connection management)

ConnectionListActivity (launcher)  →  ConnectionEditActivity  →  VncActivity
   saved connections grid              new/edit form               the actual VNC session

• SavedConnection.kt — data model + ConnectionType enum. ConnectionStore.kt saves/loads/deletes via SharedPreferences+JSON (⚠️ passwords are stored as plaintext — production apps should use EncryptedSharedPreferences).
• ConnectionListActivity.kt — the "Search Connections" grid screen. Search box, 2-column grid, tap=connect, long-press=Edit/Delete menu, "+" for a new connection. Limitation: an actual live screenshot is not used as the thumbnail; a generic monitor icon is shown instead — caching the last frame as a thumbnail is not implemented yet.
• ConnectionEditActivity.kt — Connection Type spinner (6 options), Title, VNC Server/Port, Username, Password+Keep, Show/Hide Advanced Settings (shows SSH fields when SSH_TUNNEL is selected), Copy Logcat (copies the app's own process logcat to the clipboard), a Help dialog, and Save.
• VncActivity.kt (formerly MainActivity) — no longer has an input form, it just connects directly using a SavedConnection's id, and shows a detailed error dialog before returning to the list if something fails.

⚠️ An important limitation, stated honestly

ConnectionType has 6 options in the dropdown, but only BASIC_VNC and SSH_TUNNEL actually work (see the ConnectionType.implemented flag). UltraVNC, Secure VNC over AnonTLS, VeNCrypt, and SSL Tunnel each need their own TLS/handshake protocol that hasn't been written yet. They can be selected and saved, but attempting to connect will show a clear "not supported yet" message instead of silently misbehaving. To add these types in the future, write a separate class (e.g. TlsRfbProtocol/UltraVncProtocol, similar to RfbProtocol) and route to it from VncActivity.buildSessionConfig().

Encoding architecture (Hextile/ZRLE)

hextileDecoder and zrleDecoder in RfbProtocol are created once per session and reused for the whole connection — this matters especially for the Inflater inside ZrleDecoder, because ZRLE's zlib stream is continuous across the whole session; creating a new Inflater per rect would break decompression.

Every encoding (Raw/Hextile/ZRLE) decodes into a single pixelWorkBuffer: IntArray that only grows when needed (grow-only) and is never reallocated per frame — Bitmap.setPixels(pixels, 0, w, x, y, w, h) only reads the first w*h elements based on the given stride, so extra (larger) buffer capacity is harmless.

The client sends SetEncodings in the order [ZRLE, Hextile, CopyRect, Raw] — per the RFB spec, the order of the list expresses the client's preference, so ZRLE will be used if the server supports it, otherwise it falls back step by step.

Build scaffold

app/src/main/
├── AndroidManifest.xml       # INTERNET permission + activity registrations
├── res/values/{themes,strings,colors}.xml
├── res/mipmap-anydpi-v26/ic_launcher.xml   # adaptive icon (vector, no PNG needed)
└── res/drawable/ic_launcher_foreground.xml

build.gradle.kts, settings.gradle.kts, gradle.properties   # root-level build files
gradle/wrapper/gradle-wrapper.properties

The Gradle wrapper (gradlew, gradlew.bat, and gradle-wrapper.jar) is committed to the repo and pinned to Gradle 8.7, so a fresh clone can build with ./gradlew assembleDebug without installing Gradle first (JDK 17+ and the Android SDK are still required). The CI workflow installs Gradle directly as a belt-and-braces fallback.

minSdk was raised from 24 to 26 — because the adaptive launcher icon (vector-based, mipmap-anydpi-v26/) only works on API 26+. To support API 24-25, add raster (PNG) icons under mipmap-hdpi/mdpi/xhdpi/... and set minSdk = 24 again — generating PNG assets was outside what could be done in this sandbox.

Usage (example)

val controller = VncSessionController(context, canvasView)

val sshConfig = SshTunnelManager.SshConfig(
    sshHost = "my-server.example.com",
    sshUser = "deploy",
    privateKeyPath = "/data/data/com.boffin.vncforge/files/id_ed25519",
    remotePort = 5900,
    localPort = 5901
)

controller.connect(
    VncSessionController.SessionConfig(
        useSshTunnel = true,
        sshConfig = sshConfig,
        directVncHost = "",
        directVncPort = 0,
        vncPassword = "vnc-server-password"
    )
)

To connect directly without an SSH tunnel, set useSshTunnel = false and supply directVncHost/directVncPort.

Virtual keyboard usage

Place VirtualKeyboardOverlay above the VNC canvas in your Activity/Fragment (e.g. inside a FrameLayout) and attach it to the session:

val keyboardOverlay = findViewById<VirtualKeyboardOverlay>(R.id.virtualKeyboard)
controller.attachVirtualKeyboard(keyboardOverlay)

• Esc / Tab / Enter / arrow keys — tapping sends a key-down + key-up pair (RfbProtocol.sendKeyEvent → RFB client message type 4).
• Ctrl / Alt / Shift — these are ToggleButtons, so pressing once sends key-down and stays "held", forming a combo with the next regular key press (from a hardware/soft keyboard text input, e.g. Ctrl+C, Alt+Tab); pressing again sends key-up and toggles off.
• When the session disconnects, controller.disconnect() automatically resets the toggle buttons (VirtualKeyboardOverlay.resetToggles()), so Ctrl/Alt don't stay stuck on for the next session.

CI: automated APK builds (.github/workflows/build-apk.yml)

Every push to main, every pull request, and manual runs (workflow_dispatch) trigger a GitHub Actions workflow that:

1. Checks out the repo and sets up JDK 17 (Temurin).
2. Installs Gradle directly (via gradle/actions/setup-gradle, pinned to the version in gradle/wrapper/gradle-wrapper.properties) — this means the build does not require gradlew/the wrapper jar to be committed.
3. Builds assembleDebug and assembleRelease.
4. Uploads both APKs as workflow artifacts (vnc-forge-debug, vnc-forge-release), downloadable from the Actions run page.

Release signing

The release build type is set up to use a real signing key if the following repository secrets are configured, and otherwise falls back to Android's auto-generated debug keystore so the release artifact is always a valid, installable APK for testing (not suitable for a production Play Store release without your own key):

Secret name	Description
RELEASE_KEYSTORE_BASE64	Your .jks/.keystore file, base64-encoded
RELEASE_KEYSTORE_PASSWORD	Keystore password
RELEASE_KEY_ALIAS	Key alias inside the keystore
RELEASE_KEY_PASSWORD	Password for that key alias

To generate a keystore and the base64 secret locally:

keytool -genkey -v -keystore release.jks -keyalg RSA -keysize 2048 -validity 10000 -alias vncforge
base64 -w0 release.jks > release.jks.base64   # paste this into the RELEASE_KEYSTORE_BASE64 secret

Without these secrets, assembleRelease still succeeds and produces an installable (debug-signed) APK — just not one you should publish.

Options menu, input modes, scaling, and color modes

VncActivity now has a "⋮" floating button (bottom-right, next to the keyboard toggle) that opens a dark overlay menu built by VncOptionsMenuView.kt:

Disable Extra Keys
Input Mode ▸        (Direct Touch / Touchpad / Mouse Hover)
Scaling ▸            (Fit to Screen / 1:1 Original / Zoom)
────────────────────
Disconnect
Send Keys            (Tab, Esc, Enter, arrows, F1-F12, Super, Print Screen, ...)
Send Key Again        (repeats the last key sent via Send Keys or Ctrl-Alt-Del)
Ctrl-Alt-Del
Send Text             (types a whole string into the session character by character)
────────────────────
Color Mode ▸          (24-bit True Color / 16-bit / 8-bit)
Mouse @ (x, y)        (toggles an on-screen pointer indicator)
────────────────────
Info
Input Mode Help

• InputMode.kt / ScalingMode.kt / ColorMode.kt — the three enums driving this menu.
• VncCanvasView.kt — the touch-handling and rendering logic per mode:
  • Direct Touch: finger position maps directly to the remote pointer; drag = click-and-hold.
  • Touchpad: dragging moves a persistent virtual cursor by the same delta (like a laptop trackpad), independent of where you touch; a tap clicks at the cursor's current position.
  • Mouse Hover: finger position maps directly to the pointer, but movement alone never clicks (buttonMask=0) — only an explicit tap sends a click.
  • Fit to Screen / 1:1 Original / Zoom control the base scale; two-finger pinch+pan work in Zoom (and pan alone works in Original), while Fit to Screen deliberately ignores both so the whole desktop always stays visible.
• RfbProtocol.changeColorMode() / sendCtrlAltDel() / sendText() — new protocol-level helpers. A scope note on 8-bit color: rather than implementing a true indexed-palette pipeline (which would require tracking a 256-entry color map through every decoder), 8-bit mode sends an RGB332 true-color pixel format — still exactly 1 byte/pixel, so bandwidth savings are identical, and it reuses the existing generic PixelFormat decode path with no new code. A server that only supports genuine palette-indexed 8-bit color is not covered by this mode.

Roadmap (what's still missing)

1. Native fast path — optimize pixel blitting for high-resolution/high-FPS sessions via VncEngine.fastBlitArgb8888 (C++ side).
2. Host key pinning UI — SshTunnelManager currently disables strict checking when no known_hosts file is supplied; production apps should show the user a first-connect verification dialog (TOFU model).
3. Additional connection types — UltraVNC, AnonTLS, VeNCrypt, SSL Tunnel (see the limitation noted above).
4. Multi-monitor/resize handling — handle server-side desktop resize events (pseudo-encoding).
5. Live thumbnails — cache the last frame of a session to disk so the connection list can show real screenshots.