Compare commits

...

113 Commits

Author SHA1 Message Date
9ab29a3a15 Update some packages 2025-04-24 19:07:52 +03:30
6c5fe1bc00 Added cors 2025-04-24 19:07:43 +03:30
8f9396c791 Minor UX improvement to web UI 2025-03-10 15:58:48 +03:30
83023a31a7 Minor improvements to broken images handling in web UI 2025-03-10 15:43:13 +03:30
a1aab1ac1f Change webUI font 2025-03-04 15:00:00 +03:30
bb950b97a7 Update .dockerignore
Some checks failed
release / Build and push Docker image (push) Has been cancelled
2025-02-26 14:00:26 +03:30
4ce1b29d86 Update some packages - Add some personal notes 2025-02-26 13:59:41 +03:30
b343438f9e Performance improvements on finding a random post for API and bot '/start' command
Some checks failed
release / Build and push Docker image (push) Has been cancelled
2025-02-26 13:41:43 +03:30
39fa7a8219 Update Telegram.Bot package - Update to chiseled image for docker base image
All checks were successful
release / Build and push Docker image (push) Successful in 2m38s
2025-02-08 19:05:31 +03:30
7f26e080a4 Minor improvements to web UI
Some checks failed
release / Build and push Docker image (push) Failing after 1m2s
2025-02-08 00:16:22 +03:30
787a9dd09b Performance improvements
All checks were successful
release / Build and push Docker image (push) Successful in 1m11s
2025-01-04 23:06:13 +03:30
59410c1458 Removed binary build from gitlab ci 2025-01-04 23:05:50 +03:30
4a1bee3b94 Minor fix
All checks were successful
release / Build and push Docker image (push) Successful in 1m11s
2025-01-04 17:49:42 +03:30
e842b5129f Minor fix
All checks were successful
release / Build and push Docker image (push) Successful in 2m58s
2025-01-04 17:44:45 +03:30
cc8d5cb235 Gitea actions final fix
Some checks failed
release / Build and push Docker image (push) Failing after 20s
2025-01-04 17:41:31 +03:30
aaee37c359 Gitea actions final fix
All checks were successful
release / Build and push Docker image (push) Successful in 1m34s
2025-01-04 17:34:40 +03:30
cabdb39a55 Try to fix gitea actions
All checks were successful
release / Build and push Docker image (push) Successful in 1m21s
2025-01-04 17:30:37 +03:30
e3f6c653dd Try to fix gitea actions
Some checks failed
release / Build and push Docker image (push) Failing after 2m43s
2025-01-04 17:26:19 +03:30
28a0ff65a0 build-push-action to v5
Some checks failed
release / Build and push Docker image (push) Failing after 2m33s
2025-01-04 17:21:58 +03:30
1d31d2d456 Trying to find the issue with the gitea actions
Some checks failed
release / Build and push Docker image (push) Failing after 2m43s
2025-01-04 17:15:24 +03:30
3354be62b6 Minor fixes for gitea actions
Some checks failed
release / Build and push Docker image (push) Failing after 2m53s
2025-01-04 17:10:54 +03:30
8251fdefdb Fuck it
All checks were successful
Create and publish Docker image / docker (push) Successful in 3m13s
2025-01-04 17:06:29 +03:30
d5ae55c52d TRY to fix gitea actions
Some checks failed
release / Build and push Docker image (push) Failing after 3m21s
2025-01-04 16:52:01 +03:30
69e8fae4c5 No cache
Some checks failed
release / Build and push Docker image (push) Failing after 3m42s
2025-01-04 16:39:36 +03:30
4df22690f7 Maybe fix cache?
Some checks failed
release / Build and push Docker image (push) Has been cancelled
2025-01-04 16:34:53 +03:30
d871bd311c Now
Some checks failed
release / Build and push Docker image (push) Failing after 2m37s
2025-01-04 16:24:29 +03:30
10323af4a4 Wwhat about now? 2025-01-04 16:23:13 +03:30
4fc971d1b4 TRY TO FIX CI 2025-01-04 16:20:38 +03:30
fdc5ee9728 TRY TO FIX CI 2025-01-04 16:18:57 +03:30
68895724d7 GOD DAMN DOCKER
Some checks failed
release / Build and push Docker image (push) Failing after 2m53s
2025-01-04 16:13:04 +03:30
b73118266c Minor fixes to gitea actions
Some checks failed
release / Build and push Docker image (push) Failing after 2m53s
2025-01-04 16:02:25 +03:30
9c27064237 Remove dotnet build in gitea actions 2025-01-04 15:57:07 +03:30
860385e08f Minor fixes to gitea actions
Some checks failed
release / Build .NET Application (push) Has been cancelled
release / Build and push Docker image (push) Failing after 7m50s
2025-01-04 15:51:15 +03:30
d035251258 Minor fixes to gitea actions
Some checks failed
release / Build .NET Application (push) Failing after 22s
release / Build and push Docker image (push) Has been cancelled
2025-01-04 15:48:28 +03:30
eaf440c5ae Small fixes to gitea actions
Some checks failed
release / Build .NET Application (push) Failing after 5m39s
release / Build and push Docker image (push) Failing after 8m19s
2025-01-04 15:30:32 +03:30
eb4160e207 Fix gitea artifacts and add docker cache layer
Some checks failed
release / Build .NET Application (push) Has been cancelled
release / Build and push Docker image (push) Has been skipped
2025-01-04 15:20:39 +03:30
3e6cf9ffa1 Gitea docker test 1
Some checks failed
release / Build .NET Application (push) Successful in 5m48s
release / Build and push Docker image (push) Failing after 3m27s
2025-01-04 14:41:10 +03:30
51eaccca63 Update dotnet version to 9
Some checks failed
release / check and build (push) Failing after 4m45s
2025-01-04 14:08:28 +03:30
e38e5884a1 gitea actions label fix
Some checks failed
release / check and build (push) Failing after 4m39s
2025-01-04 00:33:43 +03:30
81ecf7d1f8 Retry gitea actions
Some checks are pending
release / check and build (push) Waiting to run
2025-01-04 00:28:39 +03:30
153d9c577b Add socks proxy support 2025-01-02 16:10:37 +03:30
798a80e820 Fixed webserver prevention the rest of the program to run 2024-12-31 23:31:11 +03:30
d02d5744ff New feature: Web api and a custom webpage to show a random cat. 2024-12-29 15:57:10 +03:30
340192d7f0 Fixed database update - general improvements 2024-12-22 18:47:14 +03:30
3c80744b80 Improve code readability 2024-12-20 23:15:26 +03:30
aa2feea611 Better error handling 2024-12-20 23:03:38 +03:30
e1b4cb65f3 Merge remote-tracking branch 'origin/main' 2024-12-16 14:26:20 +03:30
24d5175e7b Add more async database functions 2024-12-16 14:26:06 +03:30
16134407a7 Update .gitlab-ci.yml file 2024-12-15 18:35:22 +00:00
dc38ce927f Finish migration to MongoDb and general improvements 2024-12-15 21:22:36 +03:30
29011e34f9 Fix extra values 2024-12-15 18:03:18 +03:30
811a54e24d Add Dockerfile and modify gitlab CI to build the docker 2024-12-15 17:36:22 +03:30
ef8c7f2ee9 Finished the migration to MongoDb (NOT TESTED) 2024-12-15 17:32:26 +03:30
fbe0d500d9 Stage 1 of migrating to MongoDb - fully added mongodb 2024-12-15 17:01:48 +03:30
e0cdcf1198 Implemented MongoDb Basics 2024-12-06 13:35:47 +03:30
4495e6b605 Added MongoDb connection string to config options 2024-12-06 13:11:47 +03:30
321f48660d Added more logging 2024-12-03 18:05:24 +03:30
3e73caade3 Removed unused code 2024-12-01 21:04:39 +03:30
2841934d92 Added additional parameters for logging 2024-12-01 21:04:04 +03:30
32c0cd4c58 Fixed custom instance 2024-12-01 21:03:21 +03:30
45e790ef1a Update download ink for pipeline artifacts 2024-11-13 21:36:11 +03:30
d35dcd9a54 Maybe fixing the diplicate image problem? 2024-11-13 21:15:14 +03:30
71be9a43e1 [NOT TESTED] - fixing the duplicate media attachment but 2024-11-11 17:56:43 +03:30
972167634f Now not accepting posts from bot accounts - improved error handling 2024-11-07 00:12:10 +03:30
79deb67848 Only send start message if the received message is from a private account 2024-11-06 20:08:30 +03:30
97b06b7b4e Fixed callback error on "send me another cat" button 2024-11-05 20:54:56 +03:30
0c0ee21049 fixed trimmed publish problem 2024-11-05 17:55:57 +03:30
58c6be0eba fixed gitlab ci 2024-11-05 17:49:17 +03:30
c2c7732552 Migrate to dotnet 9 and minor improvements 2024-11-05 17:47:17 +03:30
51747b3e3d Migrate to Telegram.Bot v22 and some minor improvements 2024-11-05 17:25:29 +03:30
0830847336 Fixed timer bug 2024-10-31 22:18:21 +03:30
51fc4cb3fa Added new code to keep the bot running 2024-10-31 15:46:56 +03:30
d4b79e693c Removed prev commit code due to incomatiblity with systemd 2024-10-31 15:42:11 +03:30
4d751ec278 Now program exits at the press of Q. 2024-10-31 15:35:05 +03:30
ae45c7aa5a Fixed minor bug 2024-10-31 15:26:49 +03:30
d680ae818d Added DB backup command
Added automated backup every 1 hour
2024-10-31 15:16:42 +03:30
4cb0ada1d0 fix typo 2024-10-31 14:54:46 +03:30
819ce8ced2 Never trust AI 2024-10-07 20:08:42 +03:30
78c69f123e Merge remote-tracking branch 'origin/main' 2024-10-07 20:02:26 +03:30
c70f1ca0cd Bug fix 2024-10-07 20:01:55 +03:30
f6e4c96507 Bug fix 2024-10-07 19:56:51 +03:30
6b3aab913c Improved fetching config data, implemented envinronmet variable support alongside .env support 2024-10-07 19:38:38 +03:30
c675824820 Added media type formatting 2024-09-25 23:21:05 +03:30
86aa430394 Added additional logging and implemented post handling if the message is already approved 2024-09-25 23:06:18 +03:30
826f862e9a revert back the code 2024-09-16 20:19:18 +03:30
7e18f211db removed unused code 2024-09-16 20:02:13 +03:30
81e24f034d Minor fix - TODO: add .env entry for it 2024-09-16 01:35:39 +03:30
0b24b7ad36 Fixed /start command issue - tested 2024-09-16 01:27:56 +03:30
ef6520e08d revert back changes 2024-09-16 00:49:38 +03:30
e14c8023d5 Refactor update handling logic in MastodonBot class to use if/else instead of switch. 2024-09-16 00:44:09 +03:30
9fd9364257 Refactor bot event handling, add OnMessage call in OnUpdate. 2024-09-16 00:37:27 +03:30
91b28d34b6 Merge remote-tracking branch 'origin/main' 2024-09-16 00:28:22 +03:30
c92442ad99 a fix to the latest code 2024-09-16 00:27:57 +03:30
79f2318a64 Merge remote-tracking branch 'origin/main' 2024-09-16 00:26:29 +03:30
5fde0d3169 Merge remote-tracking branch 'origin/main' 2024-09-16 00:08:05 +03:30
94ee45d9bd added /start message handling 2024-09-16 00:07:24 +03:30
8b758e0bf4 Update README.md 2024-09-15 10:31:51 +00:00
7e94e39a78 No need for gitea 2024-09-15 00:00:20 +03:30
c6c8f3ba3f minor fix
Some checks failed
release / check and build (push) Has been cancelled
2024-09-14 23:26:42 +03:30
c5136bd8f3 change runner to g7s 2024-09-14 23:24:40 +03:30
84e11ca251 gitea fix official example 2024-09-14 23:23:04 +03:30
1d833e03d2 gitea fix 2024-09-14 23:14:25 +03:30
fa0892a527 Gemini fix 2024-09-14 23:12:53 +03:30
b3ade03630 AI fix 2024-09-14 23:09:28 +03:30
0112c2ff24 Gitea fix
Some checks failed
Build / build (push) Failing after 0s
2024-09-14 23:05:41 +03:30
8ad0ab485c Gitea fix
Some checks failed
Create and publish a Docker image / build (push) Failing after 0s
2024-09-14 23:02:32 +03:30
567a90014d maybe ai fix 2024-09-14 22:55:46 +03:30
7d4bc246b4 Updated gitea-ci file 2024-09-14 22:53:16 +03:30
6b2e794428 Update gitea-ci 2024-09-14 22:43:56 +03:30
8e24feaa80 Merge remote-tracking branch 'origin/main' 2024-09-14 22:41:02 +03:30
393b0bc37a Merge remote-tracking branch 'origin/main' 2024-09-14 22:39:41 +03:30
064e465ca0 Merge remote-tracking branch 'origin/main' 2024-09-14 22:10:56 +03:30
9ea431c915 Added gitea ci/cd file 2024-09-14 22:05:53 +03:30
24 changed files with 1099 additions and 316 deletions

26
.dockerignore Executable file
View File

@@ -0,0 +1,26 @@
**/.dockerignore
**/.env
**/.git
**/.gitignore
**/.project
**/.settings
**/.toolstarget
**/.vs
**/.vscode
**/.idea
**/*.*proj.user
**/*.dbmdl
**/*.jfm
**/azds.yaml
**/bin
**/charts
**/docker-compose*
**/Dockerfile*
**/node_modules
**/npm-debug.log
**/obj
**/secrets.dev.yaml
**/values.dev.yaml
LICENSE
README.md
PersonalNotes

44
.gitea/workflows/gitea-ci.yml Executable file
View File

@@ -0,0 +1,44 @@
name: release
on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
env:
ASPNETCORE_ENVIRONMENT: Production
jobs:
docker:
name: Build and push Docker image
runs-on: ubuntu-latest
permissions:
packages: write
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Log in to the Container registry
uses: docker/login-action@v3
with:
registry: git.mahdium.ir
username: ${{ secrets.USERNAME }}
password: ${{ secrets.PASSWORD }}
- name: Extract Docker metadata
id: meta
uses: docker/metadata-action@v5
with:
images: git.mahdium.ir/mahdium/cats-of-mastodon-telegram-bot
- name: Build and push Docker image
uses: docker/build-push-action@v6
with:
context: .
push: true
tags: git.mahdium.ir/mahdium/cats-of-mastodon-telegram-bot:${{gitea.sha}},git.mahdium.ir/mahdium/cats-of-mastodon-telegram-bot:latest
labels: ${{ steps.meta.outputs.labels }}
#tags: ${{ steps.meta.outputs.tags }}
#labels: ${{ steps.meta.outputs.labels }}

5
.gitignore vendored Normal file → Executable file
View File

@@ -3,4 +3,7 @@ bin/
obj/
*.log
*.json
.env
.env
data/
Folder.DotSettings.user
docker-compose.yaml

72
.gitlab-ci.yml Normal file → Executable file
View File

@@ -1,21 +1,59 @@
stages:
- build
# To contribute improvements to CI/CD templates, please follow the Development guide at:
# https://docs.gitlab.com/ee/development/cicd/templates.html
# This specific template is located at:
# https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Docker.gitlab-ci.yml
variables:
DOTNET_CLI_TELEMETRY_OPTOUT: "1"
DOTNET_SKIP_FIRST_TIME_EXPERIENCE: "1"
# Build a Docker image with CI/CD and push to the GitLab registry.
# Docker-in-Docker documentation: https://docs.gitlab.com/ee/ci/docker/using_docker_build.html
#
# This template uses one generic job with conditional builds
# for the default branch and all other (MR) branches.
before_script:
- apt-get update && apt-get install -y tar
build:
docker-build:
# Use the official docker image.
image: docker:cli
stage: build
image: mcr.microsoft.com/dotnet/sdk:8.0
services:
- docker:dind
variables:
DOCKER_IMAGE_NAME: $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG
before_script:
- docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY
# All branches are tagged with $DOCKER_IMAGE_NAME (defaults to commit ref slug)
# Default branch is also tagged with `latest`
script:
- dotnet nuget add source https://pkgs.dev.azure.com/tgbots/Telegram.Bot/_packaging/release/nuget/v3/index.json -n Telegram.Bot
- dotnet restore --no-cache
- dotnet publish -c Release -r linux-x64 --self-contained true /p:PublishSingleFile=true /p:PublishTrimmed=false /p:EnableCompressionInSingleFile=true
- tar -czvf publish.tar.gz -C bin/Release/net8.0/linux-x64/publish/ .
artifacts:
paths:
- publish.tar.gz
- docker build --pull -t "$DOCKER_IMAGE_NAME" .
- docker push "$DOCKER_IMAGE_NAME"
- |
if [[ "$CI_COMMIT_BRANCH" == "$CI_DEFAULT_BRANCH" ]]; then
docker tag "$DOCKER_IMAGE_NAME" "$CI_REGISTRY_IMAGE:latest"
docker push "$CI_REGISTRY_IMAGE:latest"
fi
# Run this job in a branch where a Dockerfile exists
rules:
- if: $CI_COMMIT_BRANCH
exists:
- Dockerfile
#stages:
# - build
#
#variables:
# DOTNET_CLI_TELEMETRY_OPTOUT: "1"
# DOTNET_SKIP_FIRST_TIME_EXPERIENCE: "1"
#
#before_script:
# - apt-get update && apt-get install -y tar
#
#bin-build:
# stage: build
# image: mcr.microsoft.com/dotnet/sdk:9.0-bookworm-slim
# script:
# - dotnet nuget add source https://pkgs.dev.azure.com/tgbots/Telegram.Bot/_packaging/release/nuget/v3/index.json -n Telegram.Bot
# - dotnet restore --no-cache
# - dotnet publish -c Release -r linux-x64 --self-contained true /p:PublishSingleFile=true /p:PublishTrimmed=false /p:EnableCompressionInSingleFile=true
# - tar -czvf publish.tar.gz -C bin/Release/net9.0/linux-x64/publish/ .
# artifacts:
# paths:
# - publish.tar.gz

13
.idea/.idea.CatsOfMastodonBot.dir/.idea/.gitignore generated vendored Executable file
View File

@@ -0,0 +1,13 @@
# Default ignored files
/shelf/
/workspace.xml
# Rider ignored files
/modules.xml
/projectSettingsUpdater.xml
/.idea.CatsOfMastodonBot.iml
/contentModel.xml
# Editor-based HTTP Client requests
/httpRequests/
# Datasource local storage ignored files
/dataSources/
/dataSources.local.xml

View File

@@ -0,0 +1,8 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="UserContentModel">
<attachedFolders />
<explicitIncludes />
<explicitExcludes />
</component>
</project>

View File

@@ -0,0 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="VcsDirectoryMappings">
<mapping directory="" vcs="Git" />
</component>
</project>

21
Dockerfile Executable file
View File

@@ -0,0 +1,21 @@
FROM mcr.microsoft.com/dotnet/aspnet:9.0.1-noble-chiseled-composite AS base
USER $APP_UID
WORKDIR /app
FROM mcr.microsoft.com/dotnet/sdk:9.0.102-bookworm-slim-amd64 AS build
ARG BUILD_CONFIGURATION=Release
WORKDIR /src
COPY ["mstdnCats.csproj", "./"]
RUN dotnet restore "mstdnCats.csproj"
COPY . .
WORKDIR "/src/"
RUN dotnet build "mstdnCats.csproj" -c $BUILD_CONFIGURATION -o /app/build
FROM build AS publish
ARG BUILD_CONFIGURATION=Release
RUN dotnet publish "mstdnCats.csproj" -c $BUILD_CONFIGURATION -o /app/publish /p:UseAppHost=false
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "mstdnCats.dll"]

56
Models/ConfigData.cs Executable file
View File

@@ -0,0 +1,56 @@
namespace CatsOfMastodonBot.Models;
public class ConfigData
{
public static config fetchData()
{
// Load from .env file first
DotNetEnv.Env.Load();
// Fetch values from .env file or environment variables (fall back)
var dbName = DotNetEnv.Env.GetString("DB_NAME") ??
Environment.GetEnvironmentVariable("DB_NAME") ?? "catsofmastodon";
var mongoDbConnectionString = DotNetEnv.Env.GetString("MONGODB_CONNECTION_STRING") ??
Environment.GetEnvironmentVariable("MONGODB_CONNECTION_STRING");
var botToken = DotNetEnv.Env.GetString("BOT_TOKEN") ?? Environment.GetEnvironmentVariable("BOT_TOKEN");
var tag = DotNetEnv.Env.GetString("TAG") ?? Environment.GetEnvironmentVariable("TAG");
var channelNumId = DotNetEnv.Env.GetString("CHANNEL_NUMID") ??
Environment.GetEnvironmentVariable("CHANNEL_NUMID");
var adminNumId = DotNetEnv.Env.GetString("ADMIN_NUMID") ?? Environment.GetEnvironmentVariable("ADMIN_NUMID");
var instance = DotNetEnv.Env.GetString("CUSTOM_INSTANCE") ??
Environment.GetEnvironmentVariable("CUSTOM_INSTANCE") ?? "https://haminoa.net";
var socksProxy = DotNetEnv.Env.GetString("SOCKS_PROXY") ?? Environment.GetEnvironmentVariable("SOCKS_PROXY") ?? String.Empty;
// Check if any of the values are still null or empty
if (string.IsNullOrEmpty(botToken) || string.IsNullOrEmpty(tag)
|| string.IsNullOrEmpty(channelNumId) || string.IsNullOrEmpty(adminNumId) || string.IsNullOrEmpty(mongoDbConnectionString))
return null; // Failure if any are missing
// If all required variables are present, assign to the config
var config = new config
{
DB_NAME = dbName,
MONGODB_CONNECTION_STRING = mongoDbConnectionString,
BOT_TOKEN = botToken,
TAG = tag,
CHANNEL_NUMID = channelNumId,
ADMIN_NUMID = adminNumId,
INSTANCE = instance,
SOCKS_PROXY = socksProxy
};
return config; // Success
}
public class config
{
public string DB_NAME { get; set; }
public string BOT_TOKEN { get; set; }
public string TAG { get; set; }
public string CHANNEL_NUMID { get; set; }
public string ADMIN_NUMID { get; set; }
public string INSTANCE { get; set; }
public string MONGODB_CONNECTION_STRING { get; set; }
public string SOCKS_PROXY { get; set; }
}
}

71
Models/Post.cs Normal file → Executable file
View File

@@ -1,50 +1,39 @@
using System.Text.Json.Serialization;
using MongoDB.Bson;
using MongoDB.Bson.Serialization.Attributes;
using Telegram.Bot.Types;
namespace mstdnCats.Models
namespace mstdnCats.Models;
[BsonIgnoreExtraElements]
public class Post
{
public class Post
{
[JsonPropertyName("id")]
public required string mstdnPostId { get; set; }
[JsonPropertyName("url")]
public required string Url { get; set; }
[JsonPropertyName("account")]
public required Account Account { get; set; }
[JsonPropertyName("media_attachments")]
public required List<MediaAttachment> MediaAttachments { get; set; }
}
[JsonPropertyName("id")] public required string mstdnPostId { get; set; }
[JsonPropertyName("url")] public required string Url { get; set; }
[JsonPropertyName("account")] public required Account Account { get; set; }
public class Account
{
[JsonPropertyName("id")]
public required string AccId { get; set; }
[JsonPropertyName("username")]
public required string Username { get; set; }
[JsonPropertyName("acct")]
public required string Acct { get; set; }
[JsonPropertyName("display_name")]
public required string DisplayName { get; set; }
[JsonPropertyName("url")]
public required string Url { get; set; }
[JsonPropertyName("avatar_static")]
public required string AvatarStatic { get; set; }
}
[JsonPropertyName("media_attachments")]
public required List<MediaAttachment> MediaAttachments { get; set; }
}
public class MediaAttachment
{
[JsonPropertyName("id")]
public required string MediaId { get; set; }
[JsonPropertyName("type")]
public required string Type { get; set; }
[JsonPropertyName("url")]
public required string Url { get; set; }
[JsonPropertyName("preview_url")]
public required string PreviewUrl { get; set; }
[JsonPropertyName("remote_url")]
public required string RemoteUrl { get; set; }
public Boolean Approved { get; set; } = false;
public class Account
{
[JsonPropertyName("id")] public required string AccId { get; set; }
[JsonPropertyName("username")] public required string Username { get; set; }
[JsonPropertyName("acct")] public required string Acct { get; set; }
[JsonPropertyName("display_name")] public required string DisplayName { get; set; }
[JsonPropertyName("bot")] public required bool IsBot { get; set; }
[JsonPropertyName("url")] public required string Url { get; set; }
[JsonPropertyName("avatar_static")] public required string AvatarStatic { get; set; }
}
}
public class MediaAttachment
{
[JsonPropertyName("id")] public required string MediaId { get; set; }
[JsonPropertyName("type")] public required string Type { get; set; }
[JsonPropertyName("url")] public required string Url { get; set; }
[JsonPropertyName("preview_url")] public required string PreviewUrl { get; set; }
[JsonPropertyName("remote_url")] public required string RemoteUrl { get; set; }
public bool Approved { get; set; } = false;
}

5
PersonalNotes/faster-query.md Executable file
View File

@@ -0,0 +1,5 @@
# How did I go from 4 second on each API request to under 1 second?
### 1. Using projection to limit fields
### 2. Using proper indexes
### 3. Using $sample along with aggregation

161
Program.cs Normal file → Executable file
View File

@@ -1,64 +1,141 @@
using Microsoft.Extensions.Logging;
using System.Net;
using CatsOfMastodonBot.Models;
using CatsOfMastodonBot.Services;
using Microsoft.AspNetCore.Hosting;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
using mstdnCats.Services;
using Telegram.Bot;
using Telegram.Bot.Types;
using Telegram.Bot.Types.Enums;
public class MastodonBot
{
private static Timer _timer;
private static Timer _postFetchTimer, _backupTimer;
private static async Task Main()
{
DotNetEnv.Env.Load();
// Configure logging
using var loggerFactory = LoggerFactory.Create(builder =>
{
builder.AddConsole();
});
using var loggerFactory = LoggerFactory.Create(builder => { builder.AddConsole(); });
var logger = loggerFactory.CreateLogger<MastodonBot>();
if(!CheckEnv.IsValid()){
logger.LogCritical("Error reading envinonment variables, either some values are missing or no .env file was found");
throw new Exception("Error reading envinonment variables, either some values are missing or no .env file was found");
}
// Setup DB
var db = await DbInitializer.SetupDb(DotNetEnv.Env.GetString("DB_NAME"));
if (db == null)
// Read environment variables
var config = ConfigData.fetchData();
if (config == null)
{
logger.LogCritical("Unable to setup DB");
throw new Exception("Unable to setup DB");
logger.LogCritical(
"Error reading environment variables, either some values are missing or no .env file was found");
throw new Exception(
"Error reading environment variables, either some values are missing or no .env file was found");
}
// Setup DB
var db = await DbInitializer.SetupDb(config.MONGODB_CONNECTION_STRING, config.DB_NAME);
logger.LogInformation("DB setup done");
// Setup bot
var bot = new TelegramBotClient(DotNetEnv.Env.GetString("BOT_TOKEN"));
var me = await bot.GetMeAsync();
await bot.DropPendingUpdatesAsync();
bot.OnUpdate += OnUpdate;
logger.LogInformation($"Bot is running as {me.FirstName}.");
// Handle bot updates
async Task OnUpdate(Update update)
// Web server setup
var host = Host.CreateDefaultBuilder()
.ConfigureWebHostDefaults(webBuilder =>
{
switch (update)
webBuilder.UseKestrel(options =>
{
case { CallbackQuery: { } callbackQuery }: await HandlePostAction.HandleCallbackQuery(callbackQuery, db, bot, logger); break;
default: logger.LogInformation($"Received unhandled update {update.Type}"); break;
};
options.ListenAnyIP(5005); // Listen on port 5005
});
webBuilder.ConfigureServices(services =>
{
services.AddCors(options =>
{
options.AddDefaultPolicy(options =>
{
options.AllowAnyHeader()
.AllowAnyMethod()
.AllowAnyOrigin();
});
});
} );
ServerStartup.Serverstartup(db);
webBuilder.UseStartup<ServerStartup>();
})
.Build();
// Setup bot
TelegramBotClient bot;
if (!String.IsNullOrEmpty(config.SOCKS_PROXY))
{
WebProxy proxy = new (config.SOCKS_PROXY);
HttpClient httpClient = new (
new SocketsHttpHandler { Proxy = proxy, UseProxy = true }
);
bot = new TelegramBotClient(config.BOT_TOKEN, httpClient);
}
else
{
bot = new TelegramBotClient(config.BOT_TOKEN);
}
var me = await bot.GetMe();
await bot.DropPendingUpdates();
bot.OnMessage += OnMessage;
bot.OnUpdate += OnUpdate;
logger.LogInformation("Setup complete");
logger.LogInformation($"Bot is running as {me.FirstName} - instance: {config.INSTANCE}");
// Handle bot updates - For glass buttons functionality
async Task OnUpdate(Update update)
{
switch (update)
{
case { CallbackQuery: { } callbackQuery }:
{
// Send a new cat picture
if (callbackQuery.Data == "new_random")
{
await HandleStartMessage.HandleStartMessageAsync(callbackQuery.Message, bot, db, logger,
callbackQuery);
break;
}
// Approve or reject a post
else if (callbackQuery.Data.Contains("approve-") || callbackQuery.Data.Contains("reject-"))
{
await HandlePostAction.HandleCallbackQuery(callbackQuery, db, bot, logger);
break;
}
break;
}
default: logger.LogInformation($"Received unhandled update {update.Type}"); break;
}
// Set a timer to fetch and process posts every 15 minutes
_timer = new Timer(async _ => await RunCheck.runAsync(db, bot, DotNetEnv.Env.GetString("TAG"), logger, DotNetEnv.Env.GetString("CUSTOM_INSTANCE") ?? default), null, TimeSpan.Zero, TimeSpan.FromMinutes(15));
Console.ReadLine();
;
}
// Handle bot messages
async Task OnMessage(Message message, UpdateType type)
{
if (message.Text == "/start" && message.Chat.Type == ChatType.Private)
await HandleStartMessage.HandleStartMessageAsync(message, bot, db, logger);
else if (message.Text == "/backup" && message.Chat.Type == ChatType.Private)
await HandleDbBackup.HandleDbBackupAsync(bot, logger, config.DB_NAME, config.ADMIN_NUMID, db);
// Send a message to prompt user to send /start and recieve their cat photo only if its from a telegram user and not a channel
else if (message.Chat.Type == ChatType.Private)
await HandleStartMessage.HandleStartMessageAsync(message, bot, db, logger);
}
// Set a timer to fetch and process posts every 10 minutes
_postFetchTimer = new Timer(async _ => await RunCheck.runAsync(db, bot, config.TAG, logger, config.INSTANCE),
null, TimeSpan.Zero, TimeSpan.FromMinutes(10));
// Another timer to automatically backup the DB every 6 hour
_backupTimer =
new Timer(
async _ => await HandleDbBackup.HandleDbBackupAsync(bot, logger, config.DB_NAME, config.ADMIN_NUMID, db), null, TimeSpan.Zero, TimeSpan.FromHours(6));
// Keep the bot running
await host.RunAsync();
}
}
}

42
README.md Normal file → Executable file
View File

@@ -1,23 +1,26 @@
# Disclaimer:
This project involves scraping public data from Mastodon timelines. While this data is generally publicly accessible, it's important to note that scraping and archiving such data may have implications, especially if it involves sensitive information or terms of service violations.
This project involves scraping public data from Mastodon timelines. While this data is generally publicly accessible,
it's important to note that scraping and archiving such data may have implications, especially if it involves sensitive
information or terms of service violations.
I am not responsible for any misuse or unauthorized use of the scraped data. It is the user's responsibility to ensure compliance with all applicable laws, regulations, and terms of service when storing, archiving, or using the data.
I am not responsible for any misuse or unauthorized use of the scraped data. It is the user's responsibility to ensure
compliance with all applicable laws, regulations, and terms of service when storing, archiving, or using the data.
## Required Environment Variables
| Variable Name | Description | Default Value | Format |
|---|---|---|---|
| DB_NAME | Database file name | (Required) | Must end in *.json |
| BOT_TOKEN | Telegram bot token | (Required) | Standard Telegram bot token format |
| TAG | Mastodon timeline tag | (Required) | Text with no spaces |
| CHANNEL_NUMID | Telegram channel number ID | (Required) | Telegram channel number ID format |
| ADMIN_NUMID | Telegram bot admin/reviewer number ID | (Required) | Telegram user number ID format |
| Variable Name | Description | Default Value | Format |
|---------------|---------------------------------------|---------------|-----------------------------------------|
| DB_NAME | Database file name | (Required) | Must not have any extension, plain text |
| BOT_TOKEN | Telegram bot token | (Required) | Standard Telegram bot token format |
| TAG | Mastodon timeline tag | (Required) | Text with no spaces |
| CHANNEL_NUMID | Telegram channel number ID | (Required) | Telegram channel number ID format |
| ADMIN_NUMID | Telegram bot admin/reviewer number ID | (Required) | Telegram user number ID format |
## Optional Environment Variables
| Variable Name | Description | Default Value | Format |
|---|---|---|---|
| Variable Name | Description | Default Value | Format |
|-----------------|------------------------------|--------------------------------------------|------------|
| CUSTOM_INSTANCE | Custom Mastodon instance URL | [https://haminoa.net](https://haminoa.net) | URL format |
**Note:** All environment variables except `CUSTOM_INSTANCE` are required for the project to function.
@@ -26,25 +29,32 @@ I am not responsible for any misuse or unauthorized use of the scraped data. It
### Published Executable
1. Download the published executable for your operating system from the project releases.
2. Navigate to the directory containing the downloaded executable in your terminal.
1. Download the published executable for `Linux-x86` from
the [pipeline artifacts](https://gitlab.com/api/v4/projects/61685511/jobs/artifacts/main/raw/publish.tar.gz?job=build)
2. Navigate to the directory containing the downloaded archive in your terminal and extract the archive.
**Providing Environment Variables:**
**Using a `.env` file:**
1. Create a file named .env in the root directory of your project.
2. Add each environment variable on a separate line in the format KEY=VALUE. For example:
```
DB_NAME=my_data.json
DB_NAME=my_data
BOT_TOKEN=your_telegram_bot_token
TAG=important_data
TAG=mastodontimelinetag
CHANNEL_NUMID=1234567890
ADMIN_NUMID=9876543210
```
1. Run the following command:
```bash
dotnet run
or
./mstdnCats
```
**Remember to replace `your_telegram_bot_token`, `my_data.json`, `important_data`, `1234567890`, and `9876543210` with your actual values.**
**Remember to replace `your_telegram_bot_token`, `my_data.json`, `important_data`, `1234567890`, and `9876543210` with
your actual values.**

View File

@@ -1,21 +0,0 @@
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
namespace mstdnCats.Services
{
public class CheckEnv
{
public static Boolean IsValid(){
if (DotNetEnv.Env.GetString("DB_NAME") == null ||
DotNetEnv.Env.GetString("BOT_TOKEN") == null ||
DotNetEnv.Env.GetString("TAG") == null ||
DotNetEnv.Env.GetString("CHANNEL_NUMID") == null ||
DotNetEnv.Env.GetString("ADMIN_NUMID") == null ){
return false;
}
return true;
}
}
}

43
Services/DbInitializer.cs Normal file → Executable file
View File

@@ -1,31 +1,24 @@
using JsonFlatFileDataStore;
using MongoDB.Driver;
using mstdnCats.Models;
namespace mstdnCats.Services
{
public class DbInitializer
{
public static Task<IDocumentCollection<Post>> SetupDb(string _dbname)
{
// Setup DB
IDocumentCollection<Post>? collection = null;
if (_dbname != null)
{
try
{
// Initialize DB
var store = new DataStore($"{_dbname}.json", minifyJson: false);
collection = store.GetCollection<Post>();
}
catch
{
return Task.FromResult<IDocumentCollection<Post>>(null);
}
namespace mstdnCats.Services;
// Return collection
return Task.FromResult(collection);
}
return Task.FromResult(collection);
public class DbInitializer
{
public static Task<IMongoCollection<Post>> SetupDb(string mongoDbConnectionString, string dbName)
{
if (mongoDbConnectionString == null) throw new Exception("MongoDb connection string is null");
try
{
var client = new MongoClient(mongoDbConnectionString);
var database = client.GetDatabase(dbName).GetCollection<Post>("posts");
return Task.FromResult(database);
}
catch (Exception ex)
{
throw new Exception("Error while connecting to MongoDB: " + ex.Message);
}
}
}

33
Services/HandleDbBackup.cs Executable file
View File

@@ -0,0 +1,33 @@
using System.Text;
using Microsoft.Extensions.Logging;
using MongoDB.Bson;
using MongoDB.Driver;
using mstdnCats.Models;
using Telegram.Bot;
using Telegram.Bot.Types;
using Telegram.Bot.Types.Enums;
namespace mstdnCats.Services;
public class HandleDbBackup
{
public static async Task HandleDbBackupAsync(TelegramBotClient _bot, ILogger<MastodonBot>? logger, string dbname,
string adminId, IMongoCollection<Post> _db)
{
logger?.LogInformation("Backup requested");
try{
var json = (await _db.Find(new BsonDocument()).ToListAsync()).ToJson();
var bytes = Encoding.UTF8.GetBytes(json);
var stream = new MemoryStream(bytes);
var postCount = await _db.CountDocumentsAsync(new BsonDocument());
var caption =
$"Backup of the database: {dbname}\nCreated at {DateTime.Now:yyyy-MM-dd HH:mm:ss}\nCurrent post count: {postCount}";
await _bot.SendDocument(adminId, InputFile.FromStream(stream, "backup.json"), caption);
logger?.LogInformation("Backup sent");
}
catch(Exception ex){
logger?.LogError(ex,"Unable to backup database");
}
}
}

159
Services/HandlePostAction.cs Normal file → Executable file
View File

@@ -1,97 +1,124 @@
using JsonFlatFileDataStore;
using CatsOfMastodonBot.Models;
using Microsoft.Extensions.Logging;
using MongoDB.Driver;
using mstdnCats.Models;
using Telegram.Bot;
using Telegram.Bot.Types;
using Telegram.Bot.Types.Enums;
using Telegram.Bot.Types.ReplyMarkups;
namespace mstdnCats.Services
namespace mstdnCats.Services;
public class HandlePostAction
{
public class HandlePostAction
public static async Task HandleCallbackQuery(CallbackQuery callbackQuery, IMongoCollection<Post> _db,
TelegramBotClient _bot, ILogger<MastodonBot>? logger)
{
public static async Task HandleCallbackQuery(CallbackQuery callbackQuery, IDocumentCollection<Post> _db, TelegramBotClient _bot, ILogger<MastodonBot>? logger)
var config = ConfigData.fetchData();
// Extract media ID from callback query data
string[] parts = callbackQuery.Data.Split('-');
if (parts.Length != 2)
{
// Extract media ID from callback query data
string[] parts = callbackQuery.Data.Split('-');
if (parts.Length != 2)
logger?.LogError("Invalid callback query data format.");
return;
}
var action = parts[0];
var mediaId = parts[1];
var filter = Builders<Post>.Filter.Eq("MediaAttachments.MediaId", mediaId);
var post = await _db.Find(filter).FirstOrDefaultAsync();
if (post == null)
{
logger?.LogInformation("No matching post found.");
return;
}
// Approve the media attachment
if (action == "approve")
{
var mediaAttachment = post.MediaAttachments.FirstOrDefault(m => m.MediaId == mediaId);
if (mediaAttachment != null)
{
logger?.LogError("Invalid callback query data format.");
return;
}
string action = parts[0];
string mediaId = parts[1];
var post = _db.AsQueryable().FirstOrDefault(p => p.MediaAttachments.Any(m => m.MediaId == mediaId));
if (post == null)
{
logger?.LogInformation("No matching post found.");
return;
}
// Approve the media attachment
if (action == "approve")
{
var mediaAttachment = post.MediaAttachments.FirstOrDefault(m => m.MediaId == mediaId);
if (mediaAttachment != null)
// Check if the media attachment is already approved
if (mediaAttachment.Approved)
{
mediaAttachment.Approved = true;
await _bot.AnswerCallbackQuery(callbackQuery.Id, "Media attachment is already approved.", true);
await _bot.DeleteMessage(callbackQuery.Message.Chat.Id, callbackQuery.Message.Id);
return;
}
bool updated = await _db.UpdateOneAsync(p => p.mstdnPostId == post.mstdnPostId, post);
// Update the media attachment
var update = Builders<Post>.Update.Set("MediaAttachments.$.Approved", true);
var result = await _db.UpdateOneAsync(filter, update);
if (updated)
if (result.ModifiedCount > 0)
try
{
// Send the media attachment to the channel
await _bot.SendPhotoAsync(DotNetEnv.Env.GetString("CHANNEL_NUMID"), post.MediaAttachments.First().Url, caption: $"Post from " + $"<a href=\"" + post.Account.Url + "\">" + post.Account.DisplayName + " </a>", parseMode: ParseMode.Html
, replyMarkup: new InlineKeyboardMarkup(InlineKeyboardButton.WithUrl("View on Mastodon", post.Url)));
var allMediaAttachments = post.MediaAttachments.ToList();
await _bot.SendPhoto(config.CHANNEL_NUMID,
allMediaAttachments.First(m => m.MediaId == mediaId).Url,
$"Post from " + $"<a href=\"" + post.Account.Url + "\">" +
post.Account.DisplayName + " </a>", ParseMode.Html
, replyMarkup: new InlineKeyboardMarkup(InlineKeyboardButton.WithUrl("View on Mastodon", post.Url)));
await _bot.AnswerCallbackQueryAsync(callbackQuery.Id, "Media attachment approved and sent to channel.");
await _bot.DeleteMessageAsync(callbackQuery.Message.Chat.Id, callbackQuery.Message.MessageId);
await _bot.AnswerCallbackQuery(callbackQuery.Id,
"Media attachment approved and sent to channel.");
await _bot.DeleteMessage(callbackQuery.Message.Chat.Id, callbackQuery.Message.Id);
logger?.LogTrace($"Media attachment {mediaId} approved.");
}
else
catch (Exception e)
{
logger?.LogError($"Failed to update the media attachment {mediaId}. Record might not exist or was not found.");
logger?.LogError($"Error while sending image to the channel:{e}");
}
}
else
{
logger?.LogError($"No media attachment found with MediaId {mediaId}.");
}
}
// Reject the media attachment
else if (action == "reject")
{
// Check if the post has only one attachment, if so, do not delete it, else delete the associated attachment
if (post.MediaAttachments.Count == 1 && post.MediaAttachments.First().MediaId == mediaId)
{
await _bot.AnswerCallbackQueryAsync(callbackQuery.Id, "Post has only one attachment. No deletion performed.");
await _bot.DeleteMessageAsync(callbackQuery.Message.Chat.Id, callbackQuery.Message.MessageId);
logger?.LogTrace($"Post {post.mstdnPostId} has only one attachment. No deletion performed.");
}
else
{
post.MediaAttachments.RemoveAll(m => m.MediaId == mediaId);
await _db.UpdateOneAsync(p => p.mstdnPostId == post.mstdnPostId, post);
await _bot.AnswerCallbackQueryAsync(callbackQuery.Id, "Media attachment rejected.");
await _bot.DeleteMessageAsync(callbackQuery.Message.Chat.Id, callbackQuery.Message.MessageId);
logger?.LogTrace($"Media attachment {mediaId} removed from post {post.mstdnPostId}.");
}
logger?.LogError(
$"Failed to update the media attachment {mediaId}. Record might not exist or was not found.");
}
else
{
logger?.LogError("Invalid action specified.");
logger?.LogError($"No media attachment found with MediaId {mediaId}.");
}
}
// Reject the media attachment
else if (action == "reject")
{
// Check if the post has only one attachment, if so, do not delete it, else delete the associated attachment
if (post.MediaAttachments.Count == 1 && post.MediaAttachments.First().MediaId == mediaId)
{
await _bot.AnswerCallbackQuery(callbackQuery.Id,
"Post has only one attachment. No deletion performed.");
await _bot.DeleteMessage(callbackQuery.Message.Chat.Id, callbackQuery.Message.Id);
logger?.LogTrace($"Post {post.mstdnPostId} has only one attachment. No deletion performed.");
}
else
{
var update = Builders<Post>.Update.PullFilter("MediaAttachments",
Builders<MediaAttachment>.Filter.Eq("MediaId", mediaId));
var result = await _db.UpdateOneAsync(filter, update);
if (result.ModifiedCount > 0)
{
await _bot.AnswerCallbackQuery(callbackQuery.Id, "Media attachment rejected.");
await _bot.DeleteMessage(callbackQuery.Message.Chat.Id, callbackQuery.Message.Id);
logger?.LogTrace($"Media attachment {mediaId} removed from post {post.mstdnPostId}.");
}
else
{
logger?.LogError($"Failed to remove media attachment {mediaId} from post {post.mstdnPostId}.");
}
}
}
else
{
logger?.LogError("Invalid action specified.");
}
}
}

51
Services/HandleStartMessage.cs Executable file
View File

@@ -0,0 +1,51 @@
using Microsoft.Extensions.Logging;
using MongoDB.Driver;
using MongoDB.Driver.Linq;
using mstdnCats.Models;
using Telegram.Bot;
using Telegram.Bot.Types;
using Telegram.Bot.Types.Enums;
using Telegram.Bot.Types.ReplyMarkups;
namespace CatsOfMastodonBot.Services;
public class HandleStartMessage
{
public static async Task HandleStartMessageAsync(Message message, TelegramBotClient _bot,
IMongoCollection<Post> _db, ILogger<MastodonBot>? logger, CallbackQuery callbackQuery = null)
{
logger?.LogInformation("Start message received, trigger source: " +
(callbackQuery != null ? "Callback" : "Start command"));
// choose all media attachments that are approved
// OLD QUERY
// var mediaAttachmentsToSelect = await _db.AsQueryable()
// .Where(post => post.MediaAttachments.Any(media => media.Approved))
// .ToListAsync();
var filter = Builders<Post>.Filter.ElemMatch(post => post.MediaAttachments,
Builders<MediaAttachment>.Filter.Eq(media => media.Approved, true));
var projection = Builders<Post>.Projection
.Include(p => p.Url)
.Include(p => p.Account.DisplayName)
.Include(p => p.MediaAttachments);
var selectedPost = await _db.Aggregate().Match(filter).Project<Post>(projection).Sample(1).FirstOrDefaultAsync();
// send media attachment
await _bot.SendPhoto(message.Chat.Id,
selectedPost.MediaAttachments.FirstOrDefault(m => m.Approved).RemoteUrl,
$"Here is your cat!🐈\n" + "<a href=\"" + selectedPost.Url + "\">" +
$"View on Mastodon " + " </a>", ParseMode.Html
, replyMarkup: new InlineKeyboardMarkup()
.AddButton(InlineKeyboardButton.WithUrl("Join channel 😺", "https://t.me/catsofmastodon"))
.AddNewRow()
.AddButton(InlineKeyboardButton.WithCallbackData("Send me another one!", $"new_random")));
// answer callback query from "send me another cat" button
if (callbackQuery != null) await _bot.AnswerCallbackQuery(callbackQuery.Id, "Catch your cat! 😺");
logger?.LogInformation("Random cat sent!");
}
}

51
Services/PostResolver.cs Normal file → Executable file
View File

@@ -2,33 +2,40 @@ using System.Text.Json;
using Microsoft.Extensions.Logging;
using mstdnCats.Models;
namespace mstdnCats.Services
namespace mstdnCats.Services;
public sealed class PostResolver
{
public sealed class PostResolver
public static async Task<List<Post>?> GetPostsAsync(string tag, ILogger<MastodonBot>? logger, string instance)
{
// Get posts
var _httpClient = new HttpClient();
// Get posts from mastodon api (40 latest posts)
var requestUrl = $"{instance}/api/v1/timelines/tag/{tag}?limit=40";
var response = await _httpClient.GetAsync(requestUrl);
public static async Task<List<Post>?> GetPostsAsync(string tag, ILogger<MastodonBot>? logger, string instance = "https://haminoa.net")
// Print out ratelimit info
var remainingTime = int.Parse(response.Headers.GetValues("X-RateLimit-Remaining").First());
var resetTime = DateTime.Parse(response.Headers.GetValues("X-RateLimit-Reset").First());
var diff = resetTime - DateTime.UtcNow;
logger?.LogInformation($"Remaining requests: {remainingTime}, ratelimit reset in {diff.Hours} hours {diff.Minutes} minutes {diff.Seconds} seconds");
// Check if response is ok
if (
response.StatusCode == System.Net.HttpStatusCode.OK ||
response.Content.Headers.ContentType.MediaType.Contains("application/json") ||
(response.Headers.TryGetValues("X-RateLimit-Remaining", out var remaining) &&
int.Parse(remaining.First()) != 0)
)
{
// Get posts
HttpClient _httpClient = new HttpClient();
// Get posts from mastodon api (40 latest posts)
var response = await _httpClient.GetAsync($"{instance}/api/v1/timelines/tag/{tag}?limit=40");
// Deserialize response based on 'Post' model
return JsonSerializer.Deserialize<List<Post>>(await response.Content.ReadAsStringAsync());
}
// Print out ratelimit info
logger?.LogInformation("Remaining requests: " + response.Headers.GetValues("X-RateLimit-Remaining").First() + "time to reset: " + response.Headers.GetValues("X-RateLimit-Reset").First());
// Check if response is ok
if (
response.StatusCode == System.Net.HttpStatusCode.OK ||
response.Content.Headers.ContentType.MediaType.Contains("application/json") ||
response.Headers.TryGetValues("X-RateLimit-Remaining", out var remaining) && int.Parse(remaining.First()) != 0
)
{
// Deserialize response based on 'Post' model
return JsonSerializer.Deserialize<List<Post>>(await response.Content.ReadAsStringAsync());
}
else return null;
else
{
logger?.LogCritical("Error while getting posts: " + response.StatusCode);
return null;
}
}
}

84
Services/ProcessPosts.cs Normal file → Executable file
View File

@@ -1,44 +1,64 @@
using JsonFlatFileDataStore;
using CatsOfMastodonBot.Models;
using Microsoft.Extensions.Logging;
using MongoDB.Driver;
using MongoDB.Driver.Linq;
using mstdnCats.Models;
using Telegram.Bot;
using Telegram.Bot.Types.Enums;
using Telegram.Bot.Types.ReplyMarkups;
namespace mstdnCats.Services
{
public class ProcessPosts
{
public static async Task<List<MediaAttachment>> checkAndInsertPostsAsync(IDocumentCollection<Post> _db, TelegramBotClient _bot, List<Post> fetchedPosts, ILogger<MastodonBot>? logger)
{
// Get existing posts
var existingPosts = _db.AsQueryable().Select(x => x.mstdnPostId).ToArray();
logger?.LogInformation($"Recieved posts to proccess: {fetchedPosts.Count} - total existing posts: {existingPosts.Length}");
int newPosts = 0;
// Process posts
foreach (Post post in fetchedPosts)
{
// Check if post already exists
if (!existingPosts.Contains(post.mstdnPostId) && post.MediaAttachments.Count > 0)
{
// Send approve or reject message to admin
foreach (var media in post.MediaAttachments)
{
await _bot.SendPhotoAsync(DotNetEnv.Env.GetString("ADMIN_NUMID"), media.PreviewUrl, caption: $"<a href=\"" + post.Url + "\"> Mastodon </a>", parseMode: ParseMode.Html
, replyMarkup: new InlineKeyboardMarkup().AddButton("Approve", $"approve-{media.MediaId}").AddButton("Reject", $"reject-{media.MediaId}"));
namespace mstdnCats.Services;
}
// Insert post
await _db.InsertOneAsync(post);
newPosts++;
}
public class ProcessPosts
{
public static async Task<List<MediaAttachment>> checkAndInsertPostsAsync(IMongoCollection<Post> _db,
TelegramBotClient _bot, List<Post> fetchedPosts, ILogger<MastodonBot>? logger)
{
var config = ConfigData.fetchData();
// Get existing posts
var existingPosts = await _db.AsQueryable().Select(x => x.mstdnPostId).ToListAsync();
logger?.LogInformation(
$"Recieved posts to proccess: {fetchedPosts.Count} - total existing posts: {existingPosts.Count}");
var newPosts = 0;
// Process posts
List<Post> validPosts = new();
foreach (var post in fetchedPosts)
// Check if post already exists
if (!existingPosts.Contains(post.mstdnPostId) && post.MediaAttachments.Count > 0 &&
post.Account.IsBot == false)
{
// Send approve or reject message to admin
foreach (var media in post.MediaAttachments)
if (media.Type == "image")
try
{
await _bot.SendPhoto(config.ADMIN_NUMID, media.PreviewUrl,
$"<a href=\"" + post.Url + "\"> Mastodon </a>", ParseMode.Html
, replyMarkup: new InlineKeyboardMarkup().AddButton("Approve", $"approve-{media.MediaId}").AddButton("Reject", $"reject-{media.MediaId}"));
validPosts.Add(post);
newPosts++;
}
catch (Exception ex)
{
logger?.LogError("Error while sending message to admin: " + ex.Message + " - Media URL: " +
media.PreviewUrl);
}
}
logger?.LogInformation($"Proccesing done, stats: received {fetchedPosts.Count} posts, inserted {newPosts} new posts.");
// Return list of media attachments
var alldbpostsattachmentlist = _db.AsQueryable().SelectMany(x => x.MediaAttachments).ToList();
return alldbpostsattachmentlist;
// Insert post
if (validPosts.Count != 0)
{
await _db.InsertManyAsync(validPosts);
}
logger?.LogInformation(
$"Proccesing done, stats: received {fetchedPosts.Count} posts, inserted and sent {newPosts} new posts.");
// Return list of media attachments
var alldbpostsattachmentlist = _db.AsQueryable().SelectMany(x => x.MediaAttachments).ToList();
return alldbpostsattachmentlist;
}
}

40
Services/RunCheck.cs Normal file → Executable file
View File

@@ -1,33 +1,31 @@
using JsonFlatFileDataStore;
using Microsoft.Extensions.Logging;
using MongoDB.Driver;
using mstdnCats.Models;
using Telegram.Bot;
namespace mstdnCats.Services
namespace mstdnCats.Services;
public class RunCheck
{
public class RunCheck
public static async Task<bool> runAsync(IMongoCollection<Post> _db, TelegramBotClient _bot, string _tag,
ILogger<MastodonBot>? logger, string _instance)
{
public static async Task<bool> runAsync(IDocumentCollection<Post> _db, TelegramBotClient _bot, string _tag, ILogger<MastodonBot>? logger, string _instance = "https://haminoa.net")
// Run check
try
{
// Run check
try
{
// First get posts
var posts = await PostResolver.GetPostsAsync(_tag, logger, _instance);
// First get posts
var posts = await PostResolver.GetPostsAsync(_tag, logger, _instance);
if (posts == null)
{
logger?.LogCritical("Unable to get posts");
}
if (posts == null) logger?.LogCritical("Unable to get posts");
// Then process them
await ProcessPosts.checkAndInsertPostsAsync(_db, _bot, posts, logger);
}
catch (Exception ex)
{
logger?.LogCritical("Error while running check: " + ex.Message);
}
return true;
// Then process them
await ProcessPosts.checkAndInsertPostsAsync(_db, _bot, posts, logger);
}
catch (Exception ex)
{
logger?.LogCritical("Error while running check: " + ex.Message);
}
return true;
}
}

75
Web/ServerStartup.cs Executable file
View File

@@ -0,0 +1,75 @@
using System.Diagnostics;
using System.Reflection;
using System.Text;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Http;
using MongoDB.Driver;
using mstdnCats.Models;
namespace mstdnCats.Services;
public class ServerStartup
{
private static IMongoCollection<Post> _db;
public static void Serverstartup(IMongoCollection<Post> db)
{
_db = db;
}
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
app.UseRouting();
app.UseCors();
app.UseEndpoints(endpoints =>
{
endpoints.MapGet("/", async context =>
{
var assembly = Assembly.GetEntryAssembly();
var resourceName = "mstdnCats.Web.wwwroot.index.html"; // Full resource name
using (var stream = assembly.GetManifestResourceStream(resourceName))
{
if (stream == null)
{
context.Response.StatusCode = 500;
await context.Response.WriteAsync("Something went wrong in our side.");
return;
}
using (var reader = new StreamReader(stream, Encoding.UTF8))
{
context.Response.ContentType = "text/html";
await context.Response.WriteAsync(reader.ReadToEnd());
}
}
});
endpoints.MapGet("/api/gimme", async context =>
{
// Api endpoint
// Measure execution time
var stopwatch = Stopwatch.StartNew();
// Choose all posts media attachments that are approved
var filter = Builders<Post>.Filter.ElemMatch(post => post.MediaAttachments,
Builders<MediaAttachment>.Filter.Eq(media => media.Approved, true));
var projection = Builders<Post>.Projection
.Include(p => p.Url)
.Include(p => p.Account.DisplayName)
.Include(p => p.MediaAttachments);
var selectedPost = await _db.Aggregate().Match(filter).Project<Post>(projection).Sample(1).FirstOrDefaultAsync();
// Stop and print execution time
stopwatch.Stop();
Console.WriteLine($"Query executed in: {stopwatch.ElapsedMilliseconds} ms");
// Send as JSON
selectedPost.MediaAttachments = selectedPost.MediaAttachments
.Where(media => media.Approved)
.ToList();
await context.Response.WriteAsJsonAsync(selectedPost);
});
});
}
}

296
Web/wwwroot/index.html Executable file
View File

@@ -0,0 +1,296 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Cats of Mastodon! - Mahdium</title><style>
@import url('https://fonts.googleapis.com/css2?family=Atma:wght@300;400;500;600;700&display=swap');
body {
font-family: "Atma", system-ui;
display: flex;
flex-direction: column;
justify-content: center;
align-items: center;
min-height: 100vh;
background-color: #1e1e1e;
color: #e0e0e0;
margin: 0;
overflow-x: hidden;
}
.mastodon-title {
font-size: 2.5em;
margin-bottom: 10px;
text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.3);
text-align: center;
}
.welcome-message {
margin-bottom: 20px;
font-size: 1.2em;
line-height: 1.5;
text-align: center;
}
.welcome-message span {
font-size: 1.5em;
}
.post-container {
background-color: #282828;
border-radius: 15px;
box-shadow: 0 4px 8px rgba(0, 0, 0, 0.3);
padding: 20px;
width: 90%;
max-width: 500px;
text-align: center;
transition: transform 0.2s ease-in-out;
margin-bottom: 20px;
}
.post-container:hover {
transform: scale(1.02);
}
.post-title-inner {
margin-bottom: 10px;
font-size: 1.2em;
}
.user-name {
color: #ffb347;
font-weight: bold;
font-size: 1.2em;
}
.cat-text {
color: #bbdefb;
font-style: italic;
}
.post-image {
width: 100%;
border-radius: 10px;
margin-bottom: 15px;
object-fit: cover;
max-height: 500px;
box-shadow: 2px 2px 5px rgba(0, 0, 0, 0.2);
transition: opacity 0.3s ease;
}
.post-image.loading {
opacity: 0;
}
.image-container {
position: relative;
display: inline-block;
}
.image-loading-spinner {
display: none;
position: absolute;
top: 50%;
left: 50%;
transform: translate(-50%, -50%);
border: 4px solid #f3f3f3;
border-top: 4px solid #6364ff;
border-radius: 50%;
width: 30px;
height: 30px;
animation: spin 1s linear infinite;
}
.post-image.loading + .image-loading-spinner {
display: block;
}
@keyframes spin {
0% { transform: rotate(0deg); }
100% { transform: rotate(360deg); }
}
.button-container {
display: flex;
flex-direction: column;
gap: 10px;
width: 100%;
}
.mastodon-button,
.neutral-button {
background-color: #404040;
color: #e0e0e0;
border: none;
padding: 12px 18px;
border-radius: 8px;
cursor: pointer;
text-decoration: none;
width: 100%;
box-sizing: border-box;
transition: background-color 0.3s ease, transform 0.2s ease-in-out;
font-weight: 500;
}
.mastodon-button {
background-color: #6364ff;
}
.mastodon-button:hover {
background-color: #5253e0;
transform: scale(1.03);
}
.neutral-button:hover {
background-color: #505050;
transform: scale(1.03);
}
.loading-spinner {
display: none;
border: 6px solid #f3f3f3;
border-top: 6px solid #6364ff;
border-radius: 50%;
width: 60px;
height: 60px;
animation: spin 1s linear infinite;
margin: 20px auto;
}
.post-container.loading .loading-spinner {
display: block;
}
.post-container.loading .post-image,
.post-container.loading .button-container,
.post-container.loading .post-title-inner {
display: none;
}
.footer {
display: flex;
justify-content: space-between;
align-items: center;
padding: 10px;
font-size: 0.8em;
color: #909090;
width: 90%;
max-width: 500px;
margin: 0 auto;
}
.footer a {
color: inherit;
text-decoration: none;
}
.footer a:hover {
text-decoration: underline;
}
.footer-button {
background: none;
border: none;
padding: 0;
cursor: pointer;
}
</style>
</head>
<body>
<h2 class="mastodon-title">Cats of Mastodon!</h2>
<p class="welcome-message">Welcome to Daily Catventures! </br> Get your daily dose of purr-fectly adorable feline fun! <span></span><br>Posts gathered across Mastodon 🤝</p>
<div class="post-container">
<div class="post-content" style="display: none;">
<div class="post-title-inner">
<span class="user-name"></span><span class="cat-text">'s cat!</span>
</div>
<div class="image-container">
<img class="post-image" src="" alt="Cat Photo">
<div class="image-loading-spinner"></div>
</div>
<div class="button-container">
<a class="mastodon-button" href="" target="_blank">View on Mastodon</a>
<button class="neutral-button" onclick="loadNewPost()">Show me another cat!</button>
</div>
</div>
<div class="loading-spinner"></div>
</div>
<div class="footer">
<span>© 2024 Mahdium</span>
<a href="https://mahdium.ir" class="footer-button">mahdium.ir</a>
<a href="https://gitlab.com/mahdium/cats-of-mastodon-telegram-bot" class="footer-button">Source Code</a>
</div>
<script>
const postContainer = document.querySelector('.post-container');
const postContent = document.querySelector('.post-content');
const userNameSpan = document.querySelector('.post-content .user-name');
const postImage = document.querySelector('.post-content .post-image');
const mastodonLink = document.querySelector('.post-content .mastodon-button');
const imageLoadingSpinner = document.querySelector('.image-container .image-loading-spinner');
function loadNewPost() {
postContainer.classList.add('loading');
postContent.style.display = 'none';
postImage.classList.add('loading');
fetch('/api/gimme')
.then(response => {
if (!response.ok) {
throw new Error(`HTTP error! status: ${response.status}`);
}
return response.json();
})
.then(data => {
postContainer.classList.remove('loading');
postContent.style.display = 'block';
if (!data || !data.account || !data.media_attachments || data.media_attachments.length === 0) {
console.error("Invalid API response format:", data);
alert("Invalid data received from the server.");
postImage.classList.remove('loading');
return;
}
userNameSpan.textContent = data.account.display_name;
let imageUrl = data.media_attachments[0].remote_url;
if (imageUrl) {
imageUrl = imageUrl.replace('/original/', '/small/');
} else if (data.media_attachments[0].PreviewUrl) {
imageUrl = data.media_attachments[0].PreviewUrl;
} else {
console.warn("No RemoteUrl or PreviewUrl found, using placeholder");
postImage.src = "https://s6.uupload.ir/files/a69d5fc9e900cc51_1920_kmnr.png";
postImage.classList.remove('loading');
return;
}
postImage.onload = () => {
postImage.classList.remove('loading');
window.scrollTo(0, document.body.scrollHeight);
};
postImage.onerror = () => {
console.error("Error loading image:", imageUrl);
postImage.src = "https://s6.uupload.ir/files/a69d5fc9e900cc51_1920_kmnr.png";
loadNewPost();
postImage.classList.remove('loading');
};
postImage.src = imageUrl;
mastodonLink.href = data.url;
})
.catch(error => {
postContainer.classList.remove('loading');
postImage.classList.remove('loading');
console.error("Error fetching data:", error);
alert("Failed to load a new post. Please try again later.");
});
}
loadNewPost();
</script>
</body>
</html>

32
mstdnCats.csproj Normal file → Executable file
View File

@@ -1,17 +1,25 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net8.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<JsonSerializerIsReflectionEnabledByDefault>true</JsonSerializerIsReflectionEnabledByDefault>
</PropertyGroup>
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net9.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<JsonSerializerIsReflectionEnabledByDefault>true</JsonSerializerIsReflectionEnabledByDefault>
<DockerDefaultTargetOS>Linux</DockerDefaultTargetOS>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="DotNetEnv" Version="3.1.1" />
<PackageReference Include="JsonFlatFileDataStore" Version="2.4.2" />
<PackageReference Include="Telegram.Bot" Version="21.11.0" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="DotNetEnv" Version="3.1.1" />
<PackageReference Include="MongoDB.Driver" Version="3.3.0" />
<PackageReference Include="Telegram.Bot" Version="22.4.4" />
</ItemGroup>
<ItemGroup>
<Folder Include="Web\wwwroot\" />
</ItemGroup>
<ItemGroup>
<EmbeddedResource Include="Web\wwwroot\index.html" />
</ItemGroup>
</Project>