PHLUX is an iOS app that gives you remote, intelligent control of your computer.PHLUX is an iOS app that puts your computer in your pocket. Control, monitor, and automate tasks remotely through a fast, intuitive interface.
Architecture and Technology Stack
The PHLUX system is built as a sophisticated distributed architecture that seamlessly integrates a macOS desktop application with an iOS mobile app through a robust backend infrastructure. The macOS application serves as the primary AI engine, built using PyQt6 for the user interface and Anthropic's Claude AI (specifically Claude 4.5 Sonnet) as the core intelligence. The desktop app utilizes Python with asyncio for efficient concurrent operations, and implements a powerful agent loop that continuously processes user requests, executes tools, and returns intelligent responses. The backend communication layer is powered by Flask, which runs multiple microservices including a Remote Command Server , Workout Tracker, Project Manager, Enhanced AI Dashboard, and Stock Trading Backend. All of these services are reachable globally through Ngrok tunnels, enabling the iOS app to communicate with the Mac server from anywhere in the world. The iOS app is built entirely in SwiftUI using modern declarative UI patterns, and communicates with the backend exclusively through RESTful HTTP APIs, sending JSON payloads for commands and receiving structured responses.
iOS App Features and Capabilities
The PHLUX iOS app is a comprehensive personal assistant that provides voice-controlled access to the AI with a beautiful, modern interface featuring gradient backgrounds and smooth animations. The main interface includes an animated voice visualizer that responds to audio levels, push-to-talk and conversational mode recording options, real-time response polling from the server, screenshot upload capability (users can take photos or upload from library to ask AI questions about images), and a settings panel for configuring server URLs and API tokens. The app includes seven specialized modules accessible from the home screen: Voice Command Module (the main AI interface with speech-to-text via iOS recognition and text-to-speech playback of responses), Workout Tracker with real-time pose estimation using Vision framework and AVFoundation camera access for rep counting, form analysis with keypoint tracking (shoulders, elbows, wrists, hips, knees, ankles), exercise planning, workout history, progress analytics, rest timers, and haptic feedback for rep completion. The Meal Prep Module provides comprehensive nutrition tracking with daily macro dashboards (calories, protein, carbs, fats vs. goals), meal planning with calendar view, recipe search powered by Google Custom Search API with automatic web scraping for ingredients/instructions/nutrition data using BeautifulSoup, saved recipes library with image cards, grocery list management with purchase tracking and category organization, and AI-powered meal suggestions. The Stock Trading Module delivers real-time stock market data via Yahoo Finance API (yfinance), portfolio management with holdings tracking and P/L calculations, watchlist with price alerts, AI-powered stock picks and analysis, market overview with major indices, and persistent data storage using SQLite.
Custom AI Computer Control Tools
One of the most powerful features of PHLUX is its extensive suite of custom tools that give the AI complete control over the Mac computer. At the foundation is the ComputerTool, built on Anthropic's computer use API, which enables the AI to take screenshots, move the mouse, click anywhere on screen, type text, and press keyboard shortcuts using cliclick and pyautogui libraries. The BashTool allows the AI to execute any terminal command, install software via Homebrew, run scripts, and manage system processes. The EditTool gives the AI the ability to read, write, and modify files using advanced string replacement algorithms. Beyond these core tools, PHLUX includes highly specialized custom tools: the Cambium Grow Tool connects to a custom grow room management system, allowing the AI to monitor sensors (temperature, humidity, CO2), control equipment (lights, fans, pumps), capture timelapse images, run AI assessments on plant health, and generate detailed analytics reports. The Project Manager Tool is an AI-powered Motion.ai-style task management system with automatic scheduling, calendar integration, intelligent prioritization, natural language task creation, team collaboration features, and predictive analytics for workload optimization. The Calendar Tool provides deep integration with macOS Calendar/EventKit for creating events, checking availability, and time blocking. The ClickUp Tool integrates with the ClickUp project management API for syncing tasks externally. The Web Search Tool enables real-time Google searches for current information. The DALL-E Tool generates AI images on demand. The PDF Generator Tool creates beautiful, professional PDFs with charts, tables, and formatting using matplotlib and reportlab libraries.
Backend Services and API Integration
The PHLUX backend is a sophisticated multi-service architecture that handles all business logic and data persistence. The Remote Command Server is the central hub running on Flask, exposing endpoints for command processing (receives text commands from iOS with authentication tokens, injects them into the Mac app via Qt signals, supports silent mode for stock/meal apps to suppress TTS), AI response handling (GET endpoint for iOS to poll for responses, POST endpoint for Mac app to push responses, includes source tagging to route responses to correct iOS module), screenshot management (handles screenshot uploads from iOS with base64 encoding, prevents echo loops with origin flags), direct text-to-speech for meditation guidance, and recipe search (Google Custom Search integration that queries the API with the configured CSE ID, scrapes recipe pages for schema.org Recipe JSON-LD markup, extracts ingredients/instructions/prep time/cook time/servings/nutrition, handles both structured data and HTML parsing fallbacks, returns complete recipe objects with images). The Workout Tracker Backend provides a comprehensive SQLite database schema with tables for exercises, workout sessions, exercise sets with rep data, pose keypoints, form analysis results, progress tracking, and workout templates. API endpoints include exercise listings, workout session management, set logging with reps/weight/rest time, real-time rep counting using pose keypoint analysis with angle calculations for different exercise types like squats, bench press, and bicep curls, form analysis with feedback, historical progress data for charts and 1RM calculations, and pre-built workout plan management.
