AI-Based Desktop Workflow Analysis and Automated Task Generation System
DOI:
https://doi.org/10.46647/icetetas096Keywords:
Artificial Intelligence, Machine learning, Desktop Workflow Automation, Screen Activity Analysis, Pattern Recognition, Computer Vision, Optical Character Recognition(OCR), Process Mining, Task Automation, Privacy-Preserving Systems, Productivity Analytics, Script Generation.Abstract
An application for screen recording and AI-based automation solutions designed to protect user privacy, the Stealth Python App for Screen Recording and AI-Based Auto Solutions is a desktop application that helps users to become more productive by detecting and automating repetitive tasks performed on the user's computer. Many professionals spend a large amount of time performing repetitive tasks such as entering data, filling out forms and navigating their computer, ultimately reducing their overall productivity. This application will record the user’s computer screen activity, with the user's permission, while also processing all data locally to maintain complete privacy and security for all users. The system is built using a Python Flask backend and a dashboard built on React. The system will take screenshots of the applications that are used, show what applications are being used and show how the user interacts with the screen (mouse and keyboard).
The system will use AI techniques, such as machine learning, computer vision and OCR, to analyse the recorded data, identifying the repetitive workflows and behaviours exhibited by the user. Once these behaviours or patterns have been identified, the system will automatically generate executable scripts (using Py Auto GUI) to replicate those activities, thereby reducing the manual effort of the user. The application will include the following security features: encrypted data; role-based access control; configurable data privacy features such as application blacklisting and screen masking; and an interactive dashboard providing real-time data analytics. By integrating intelligent pattern recognition with the secure, local processing of data, the application will provide users with insight into their workflow and allow them to automate time-consuming tasks. This is anticipated to produce a more efficient business process for the user, eliminate a large number of repetitive tasks, and significantly increase the productivity of the user, while also preserving the user’s right to privacy and the integrity of their data.