Link Harvester is a simple app which collects links from users. Any user can submit links which are validated and stored by the application. Users can see the submitted (links/domains) and search, sort those data. The results are displayed in a paginated table.
Here's the required technologies are used for the project
- PHP v8.2
- Laravel v10
- Alpine Js v3.x
- jQuery v3.7
- Bootstrap v5
- jQuery Datatable v1.36
The running environment tools or libs
- Docker v4.2
- Supervisor v4.x
Before you begin, ensure you have the following prerequisites installed on your system: Docker Docker Compose Git (Optional for cloning the repository)
If you haven't already, you can clone this repository using Git:
-
git clone https://github.com/misujon/link-hervester-app.git -
cd link-hervester-app
This project uses Docker for easy setup. Follow these steps to get the application up and running:
Build the Docker containers:
-
docker-compose build -
docker-compose up -d
Laravel provides a powerful job processing system called "queues," which can be managed using Supervisor. Here's how to set up Supervisor for Laravel workers within the Docker environment:
Enter the workspace container:
docker-compose exec app bash
Copy the laravel-worker.conf file to the Supervisor config folder:
cp /var/www/laravel-worker.conf /etc/supervisor/conf.d/laravel-worker.conf
Now if there's no supervisor installed follow those steps otherwise skip.
apt-get updateapt-get install -y supervisor
Instruct Supervisor to read and update its configuration to include the new .conf file:
supervisorctl rereadsupervisorctl update
Start the Laravel worker process
supervisorctl start laravel-worker:*
That's it! Supervisor is now set up to manage your Laravel worker processes.
After successfully installing the project now the project can be accessible here on 8000 port with localhost
http://localhost:800
For more easier to access database here's the adminer package also included. To check the database need to go here.
http://localhost:8080
Here's the default DB credentials. Those can be changeable as per the configuration.
username: rootpassword: mysecretpassword
- On home page all the domains are fetch through datatable with the soring and searching options. Here all the domains will be showing with the Domain links button. For each domain has some links stored to the links table and how many links are there it also mentioned into the table as Available Links
- There's another page called Add Urls. where an user can add their links. The adding process are based on some criterias are,
- After submitting the urls first it seperated all the links domains and filter them and keep the unique domains only.
- Now by linking the domain id all the links are chunked with the particular domain.
- And after processing those insert all the domains and links in a bulk inserting process.
- All those process are completing through the laravel batch jobs feature where all the links data chunked as 1000 records and executes them to the batch.
- With this process those all the inserting operation is gone to the job queue and the queue has been executed through supervisor and the data are saving throughuly.
- And finally those links are stored based on their domain. Those can be accessible here. where I fetched the links of domain: www.sample.com
Note: I've stored the domain and subdomains as a single domain, it also can be sorted as only domain.
For more information on Laravel and Docker, please refer to the official documentation:
This README should provide clear instructions for cloning, installing, and configuring your Laravel application with Docker and Supervisor. Make sure to keep your Laravel-worker.conf file up-to-date and refer to Supervisor's documentation for advanced configuration options.
Feel free to ask me anything about the project or if there's having any issue. Here's I'm available with whatsapp/telegram