Pascal M
Hello Dify Community,
I'm excited to propose a new feature for Dify: the
Background Service Integration
. This integration allows users to schedule and monitor background tasks within Dify, such as dataset syncing, health checks, backups, and webhooks processing, as well as extend it to AI-related tasks in the future.
Key Features:
  1. Frontend Monitoring Page
    : Users can access a dedicated page within the Dify frontend to monitor ongoing and scheduled tasks. This page will also include the ability to schedule tasks, select supported handlers, set parameters, and filter tasks using tags.
  2. Job List and Detail Pages
    : The integration will provide a list page displaying all scheduled jobs and a detail page for each job, containing configuration details, previous and upcoming runs, logs, and health status to show how many jobs failed in the last few hours.
  3. Recurring Task Support
    : Users will be able to set recurring tasks, including those using crontab format, and see the next execution time for each recurring job, simplifying scheduling and planning.
  4. Integrated Core Tasks
    : The integration will support scheduling Dify's integrated core tasks and calling webhooks, making it a comprehensive solution for managing various types of background jobs.
  5. Default API for Schedulable Services
    : A default API will be provided for "schedulable services" with endpoints for scheduling events, fetching updates and logs, and performing health checks. Additionally, support for checking which features are supported by a service will be included to ensure seamless integration.
Enhanced Features:
  • Configurable Parameters
    : Users can configure the integration with specific parameters like API keys and host details, providing flexibility and customization for different use cases.
  • Future AI Task Integration
    : While the initial focus is on core tasks, the integration will be designed to seamlessly extend to AI-related tasks in the future, such as scheduling model training and analysis jobs.
Why Background Service Integration?
  1. Centralized Task Management
    : Users can manage all background tasks within Dify, streamlining scheduling, monitoring, and troubleshooting processes.
  2. Improved Visibility and Control
    : The frontend monitoring page and detailed job information offer improved visibility into ongoing tasks and enhance user control over task scheduling and execution