Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

First demo in NOKIA #29

Merged
merged 58 commits into from
Apr 11, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
1431ab9
working frontend deploy
twix20 Mar 29, 2018
8dcdb88
deploy to heroku via shell .sh
twix20 Mar 29, 2018
7ac55c2
fetching projects based on rxjs subject
bartoszpogoda Mar 29, 2018
f0942b3
fixed major problem with duplicating DashboardComponent
bartoszpogoda Mar 29, 2018
36c32c6
backend dockerignore fix
bartoszpogoda Mar 30, 2018
ddc1a39
Posibly fix for backend build
twix20 Apr 3, 2018
6625c7c
Travis docker-compose build
twix20 Apr 3, 2018
8709eb6
x
twix20 Apr 3, 2018
ffdfcc9
Travis build to docker image repository
twix20 Apr 3, 2018
6fdfad5
Merge branch 'deployment' into develop
twix20 Apr 3, 2018
a291e94
Working squashed Travis deployment to heroku
twix20 Apr 4, 2018
39ca3ad
Speed up travis
twix20 Apr 4, 2018
109483e
Autofac + CRON
twix20 Apr 4, 2018
ade5fad
Fetch data every 4 minutes
twix20 Apr 4, 2018
966484a
DB memory fix, Stages get status in backend
Sjantos Apr 4, 2018
b8227c3
Added await to ProjectService
Sjantos Apr 5, 2018
ced4d81
Cleaner + transactional pipelines swap
twix20 Apr 5, 2018
a846443
Dont deploy pull_requests
twix20 Apr 5, 2018
b365555
Merge pull request #20 from nokia-wroclaw/CRON-data-fetch
bartoszpogoda Apr 5, 2018
a5dc1b5
Merge pull request #19 from nokia-wroclaw/fix/cleaner-db-leak
Sjantos Apr 5, 2018
2f5b13b
Removed Jobs from stages
Sjantos Apr 5, 2018
d25c5af
adjusted front for new project data format
bartoszpogoda Apr 5, 2018
e39a4e2
build front --prod in docker
twix20 Apr 5, 2018
e7b5bcb
Merge pull request #21 from nokia-wroclaw/fix/front-prod-build
twix20 Apr 5, 2018
a3cf724
Discovery branches number of pipelines
Sjantos Apr 6, 2018
752fee6
Changes suggested in #22 pull request
Sjantos Apr 8, 2018
bb94f3c
PR #22 fix refactor
Sjantos Apr 8, 2018
1577379
[ci skip]
Sjantos Apr 8, 2018
a647635
Update .travis.yml Deploy only push
twix20 Apr 8, 2018
c84400a
Update DynamicPipelinePanelRepository.cs
Sjantos Apr 8, 2018
3948e53
Merge pull request #22 from nokia-wroclaw/discoverPipelines
Sjantos Apr 8, 2018
e51db1f
travis test frontend without e2e and lint cuz not passing
twix20 Apr 8, 2018
dfe5c14
dynamic panel styles, defined new panel type in client, temporarily d…
bartoszpogoda Apr 8, 2018
efd149e
Merge pull request #24 from nokia-wroclaw/front-last-pipelines-panel
bartoszpogoda Apr 8, 2018
a57daa9
Separated static and dynamic pipeliens in project
Sjantos Apr 8, 2018
653058c
Merge branch 'separate' of github.com:nokia-wroclaw/innovativeproject…
Sjantos Apr 8, 2018
1200ee9
Merge pull request #23 from nokia-wroclaw/feature/travis-front-unittest
twix20 Apr 8, 2018
fdbb15e
combobox for dataProviderName in project configuration
SebastianSzkoluda Apr 9, 2018
8e487b2
done with project title in backend and frontend
SebastianSzkoluda Apr 9, 2018
881a03d
Merge pull request #26 from nokia-wroclaw/supported-providers
SebastianSzkoluda Apr 9, 2018
f6ed949
front end adjustment for new project data model
bartoszpogoda Apr 9, 2018
72fd937
extracted pipeline display logic, displaying latest pipelines
bartoszpogoda Apr 9, 2018
d183a6f
Merge branch 'develop' into separate
bartoszpogoda Apr 9, 2018
9145a8b
Merge pull request #27 from nokia-wroclaw/separate
Sjantos Apr 9, 2018
b3266ca
.gitignore .vscode [ci skip]
twix20 Apr 10, 2018
281fe9c
.gitignore .vscode [ci skip]
twix20 Apr 10, 2018
d9392fd
autocomplete when typing branch name in static panel configuration
bartoszpogoda Apr 10, 2018
5ba24fe
redirect when cancel button pressed in panel config
bartoszpogoda Apr 10, 2018
3937423
Update project after inserting new panel
twix20 Apr 10, 2018
b4fbb35
Merge pull request #28 from nokia-wroclaw/branch-name-autocomplete
twix20 Apr 10, 2018
c490862
Hangifre more often JobExpirationCheckInterval
twix20 Apr 10, 2018
fdf350e
Merge branch 'develop' of https://github.com/nokia-wroclaw/innovative…
twix20 Apr 10, 2018
b17a853
Fixed memory leak XDDD Updated dependencies
twix20 Apr 10, 2018
8784920
Removed unnecessary lines [ci skip]
twix20 Apr 10, 2018
aec5bb1
Fixed not updating static panel
Sjantos Apr 10, 2018
82be20d
Update README.md [ci skip]
Sjantos Apr 10, 2018
8ef9e13
triger ci
twix20 Apr 10, 2018
c52fa00
hangfire in production
twix20 Apr 11, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
*.sh text eol=lf
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -318,3 +318,5 @@ ASALocalRun/

# MSBuild Binary and Structured Log
*.binlog

.vscode
66 changes: 46 additions & 20 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,27 +4,53 @@ dotnet: 2.0.0
sudo: required
notifications:
slack: innoprojectsummer2018:Me0KNpSZ1ES1OnWrwTUoD2Y7
addons:
chrome: stable
apt:
sources:
- google-chrome
packages:
- google-chrome-stable
- google-chrome-beta
env:
- DOCKER_COMPOSE_VERSION=1.20.1
before_install:
- export CHROME_BIN=chromium-browser
- export DISPLAY=:99.0
- sh -e /etc/init.d/xvfb start
- sudo rm /usr/local/bin/docker-compose
- curl -L https://github.com/docker/compose/releases/download/${DOCKER_COMPOSE_VERSION}/docker-compose-`uname -s`-`uname -m` > docker-compose
- chmod +x docker-compose
- sudo mv docker-compose /usr/local/bin
- docker-compose --version
- docker ps
stages:
- build_all
- test
- name: deploy_images_to_registry
if: (branch = develop) AND (type IN (push))
- name: deploy_to_heroku
if: (branch = develop) AND (type IN (push))
jobs:
include:
- stage: frontendBuild
script:
- cd src/Dashboard.WebApi/developersDashboardFrontEnd
- npm install
- npm build
- npm test
- stage: backendBuild
- stage: build_all
script: docker-compose -f ./src/docker-compose.yml build dashboard.webapi
- script: docker-compose -f ./src/docker-compose.yml build dashboard.web

- stage: test
script:
- dotnet restore ./src
- dotnet build ./src/Dashboard.sln
- docker-compose -f ./src/docker-compose.tests.yml run --rm webapi.unittests
- script: docker-compose -f ./src/docker-compose.tests.yml run --rm frontend.test npm run test
#- script: docker-compose -f ./src/docker-compose.tests.yml run --rm frontend.test npm run e2e
#- script: docker-compose -f ./src/docker-compose.tests.yml run --rm frontend.test npm run lint

- stage: deploy_images_to_registry
script:
- docker login --username=$DOCKER_USERNAME --password=$DOCKER_PASSWORD
- docker-compose -f ./src/docker-compose.yml build dashboard.webapi
- docker-compose -f ./src/docker-compose.yml push dashboard.webapi
- script:
- docker login --username=$DOCKER_USERNAME --password=$DOCKER_PASSWORD
- docker-compose -f ./src/docker-compose.yml build dashboard.web
- docker-compose -f ./src/docker-compose.yml push dashboard.web

- stage: deploy_to_heroku
script:
- docker login --username=_ --password=$HEROKU_AUTH_TOKEN registry.heroku.com
- docker pull registry.hub.docker.com/randomdocker729/cidashboard-webapi:latest
- docker tag registry.hub.docker.com/randomdocker729/cidashboard-webapi:latest registry.heroku.com/cidasherapi/web
- docker push registry.heroku.com/cidasherapi/web
- script:
- docker login --username=_ --password=$HEROKU_AUTH_TOKEN registry.heroku.com
- docker pull registry.hub.docker.com/randomdocker729/cidashboard-frontend:latest
- docker tag registry.hub.docker.com/randomdocker729/cidashboard-frontend:latest registry.heroku.com/cidasher/web
- docker push registry.heroku.com/cidasher/web
27 changes: 26 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1 +1,26 @@
# innovativeproject-dev-dashboard [![Build Status](https://travis-ci.org/nokia-wroclaw/innovativeproject-dev-dashboard.svg?branch=develop)](https://travis-ci.org/nokia-wroclaw/innovativeproject-dev-dashboard)
# innovativeproject-dev-dashboard
[![Build Status](https://travis-ci.org/nokia-wroclaw/innovativeproject-dev-dashboard.svg?branch=develop)](https://travis-ci.org/nokia-wroclaw/innovativeproject-dev-dashboard)
---------

## Goal
The goal of this project is to build an app, which will show current states of CI systems for repositories on source control systems.


## Description
This app is designed to be viewed on TVs or other displays which are not used at the moment. It allows to configure static panels, which always shows given branch or dynamic panels for discovering newest pipelines/branches. It should be able to track many projects in many CI systems. Also, writing own plugins to make this app able to deal with new CI systems is supported.

At this moment, only [GitLab](https://gitlab.com) is supported.

### Technologies
Project is divided into frontend and backend. Technologies in use

| FrontEnd | BackEnd | Both |
| :--------------------:|:------------------:|:-----------------:|
| [Angular](https://angular.io/)| [Autofac](https://autofac.org/)| [Docker](https://www.docker.com/)|
| [Angular Material](https://material.angular.io/)| [Hangfire](https://www.hangfire.io/)| [Swagger](https://swagger.io/)|
| [Bootstrap](https://getbootstrap.com/)| [RestSHarp](http://restsharp.org/)| [Heroku](https://www.heroku.com/)|
| [angular2gridster](https://github.com/swiety85/angular2gridster)| [.NET Core 2.0](https://docs.microsoft.com/pl-pl/aspnet/core/)||

## Deployment
After push to **develop** branch and **successful** Travis build, applications will be deployed to Heroku and can be accessed here:
[Online application](http://cidasher.herokuapp.com/)
2 changes: 1 addition & 1 deletion src/.dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,4 @@ docker-compose.yml
docker-compose.*.yml
*/bin
*/obj
node_modules
**/node_modules
37 changes: 37 additions & 0 deletions src/Dashboard.Application/CronJobs.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
using System.Linq;
using System.Threading.Tasks;
using Dashboard.Application.Interfaces.Services;
using Hangfire;

namespace Dashboard.Application
{
public static class CronJobs
{
public static void Register()
{
//Fetch Data Panels Project
RecurringJob.AddOrUpdate<EnqueueFetchProjectsCiDataJob>("fetch-cidata-projects",
j => j.EnqueueFetching(), EnqueueFetchProjectsCiDataJob.CronExpression);
}
}

public class EnqueueFetchProjectsCiDataJob
{
public static readonly string CronExpression = "*/4 * * * *";

private readonly IPanelService _panelService;

public EnqueueFetchProjectsCiDataJob(IPanelService panelService)
{
_panelService = panelService;
}

public async Task EnqueueFetching()
{
var activeProjects = (await _panelService.GetActiveProjectIds()).ToList();

activeProjects.ForEach(projectId =>
BackgroundJob.Enqueue<IProjectService>(s => s.UpdateCiDataForProjectAsync(projectId)));
}
}
}
6 changes: 6 additions & 0 deletions src/Dashboard.Application/Dashboard.Application.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,12 @@
</PropertyGroup>

<ItemGroup>
<PackageReference Include="Autofac" Version="4.6.2" />
<PackageReference Include="Autofac.Configuration" Version="4.0.1" />
<PackageReference Include="Autofac.Extensions.DependencyInjection" Version="4.2.2" />
<PackageReference Include="HangFire" Version="1.6.19" />
<PackageReference Include="HangFire.Autofac" Version="2.3.1" />
<PackageReference Include="Hangfire.MemoryStorage.Core" Version="1.4.0" />
<PackageReference Include="RestSharp" Version="106.2.1" />
</ItemGroup>

Expand Down
224 changes: 112 additions & 112 deletions src/Dashboard.Application/GitLabApi/GitLabClient.cs
Original file line number Diff line number Diff line change
@@ -1,112 +1,112 @@
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using System.Web;
using Dashboard.Application.GitLabApi.Models;
using RestSharp;
using RestSharp.Extensions;
namespace Dashboard.Application.GitLabApi
{
//https://github.com/emilianoeloi/gitlab-ci-dashboard/blob/master/src/gitlab.js
public class GitLabClient
{
private RestClient Client { get; }
public GitLabClient(string apiHost, string apiKey)
{
// Convert snake_casing to CamelCasing
SimpleJson.SimpleJson.CurrentJsonSerializerStrategy = new SnakeJsonSerializerStrategy();
Client = new RestClient
{
Authenticator = new ApiTokenAuthenticator(apiKey),
BaseUrl = new Uri($"{apiHost}/api/v4/")
};
}
public async Task<Pipeline> GetPipelineById(string projectId, string pipelineId)
{
var r = await GetPipelines(projectId, pipelineId : pipelineId);
return r.FirstOrDefault();
}
public async Task<Pipeline> GetPipelineByBranch(string projectId, string pipelineBranch)
{
var r = await GetPipelines(projectId, branchName : pipelineBranch);
return r.FirstOrDefault();
}
public async Task<IEnumerable<Pipeline>> GetPipelines(string projectId, string pipelineId = "", int numberOfPipelines = 20, string branchName = "")
{
var request = new RestRequest("projects/{projectId}/pipelines/{pipelineId}", Method.GET);
request.AddUrlSegment("projectId", HttpUtility.UrlEncode(projectId));
request.AddUrlSegment("pipelineId", pipelineId);
request.AddQueryParameter("per_page", numberOfPipelines.ToString());
if (!string.IsNullOrEmpty(branchName))
request.AddQueryParameter("ref", branchName);
var r = await Client.ExecuteTaskAsync<List<Pipeline>>(request);
return r.Data;
}
public async Task<Branch> GetBranch(string projectId, string branchName)
{
var r = await GetBranches(projectId, branchName : branchName);
return r.FirstOrDefault();
}
public async Task<IEnumerable<Branch>> GetBranches(string projectId, string branchName = "")
{
var request = new RestRequest("projects/{projectId}/repository/branches/{branchName}", Method.GET);
request.AddUrlSegment("projectId", HttpUtility.UrlEncode(projectId));
request.AddUrlSegment("branchName", branchName);
var r = await Client.ExecuteTaskAsync<List<Branch>>(request);
return r.Data;
}
public async Task<Commit> GetCommitBySHA(string projectId, string commitSHA)
{
var r = await GetCommits(projectId, commitSHA : commitSHA);
return r.FirstOrDefault();
}
public async Task<IEnumerable<Commit>> GetCommits(string projectId, string commitSHA = "")
{
var request = new RestRequest("projects/{projectId}/repository/commits/{commitSHA}", Method.GET);
request.AddUrlSegment("projectId", HttpUtility.UrlEncode(projectId));
request.AddUrlSegment("commitSHA", commitSHA);
var r = await Client.ExecuteTaskAsync<List<Commit>>(request);
return r.Data;
}
public async Task<IEnumerable<Job>> GetJobs(string projectId, string pipelineId)
{
var request = new RestRequest("projects/{projectId}/pipelines/{pipelineId}/jobs", Method.GET);
request.AddUrlSegment("projectId", HttpUtility.UrlEncode(projectId));
request.AddUrlSegment("pipelineId", pipelineId);
request.AddQueryParameter("per_page", "10000");//Arbitrary value
var r = await Client.ExecuteTaskAsync<IEnumerable<Job>>(request);
return r.Data;
}
public async Task<IEnumerable<Branch>> SearchForBranchInProject(string projectId, string branchPartialName)
{
var request = new RestRequest("projects/{projectId}/repository/branches?search={branchPartialName}", Method.GET);
request.AddUrlSegment("projectId", HttpUtility.UrlEncode(projectId));
request.AddUrlSegment("branchPartialName", branchPartialName);
request.AddQueryParameter("per_page", "10000");
var r = await Client.ExecuteTaskAsync<List<Branch>>(request);
return r.Data;
}
}
}
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using System.Web;
using Dashboard.Application.GitLabApi.Models;
using RestSharp;
using RestSharp.Extensions;

namespace Dashboard.Application.GitLabApi
{
//https://github.com/emilianoeloi/gitlab-ci-dashboard/blob/master/src/gitlab.js
public class GitLabClient
{
private RestClient Client { get; }

public GitLabClient(string apiHost, string apiKey)
{
// Convert snake_casing to CamelCasing
SimpleJson.SimpleJson.CurrentJsonSerializerStrategy = new SnakeJsonSerializerStrategy();

Client = new RestClient
{
Authenticator = new ApiTokenAuthenticator(apiKey),
BaseUrl = new Uri($"{apiHost}/api/v4/")
};
}

public async Task<Pipeline> GetPipelineById(string projectId, string pipelineId)
{
var r = await GetPipelines(projectId, pipelineId : pipelineId);
return r.FirstOrDefault();
}

public async Task<Pipeline> GetPipelineByBranch(string projectId, string pipelineBranch)
{
var r = await GetPipelines(projectId, branchName : pipelineBranch);
return r.FirstOrDefault();
}

public async Task<IEnumerable<Pipeline>> GetPipelines(string projectId, string pipelineId = "", int numberOfPipelines = 20, string branchName = "")
{
var request = new RestRequest("projects/{projectId}/pipelines/{pipelineId}", Method.GET);
request.AddUrlSegment("projectId", HttpUtility.UrlEncode(projectId));
request.AddUrlSegment("pipelineId", pipelineId);

request.AddQueryParameter("per_page", numberOfPipelines.ToString());

if (!string.IsNullOrEmpty(branchName))
request.AddQueryParameter("ref", branchName);

var r = await Client.ExecuteTaskAsync<List<Pipeline>>(request);
return r.Data;
}

public async Task<Branch> GetBranch(string projectId, string branchName)
{
var r = await GetBranches(projectId, branchName : branchName);
return r.FirstOrDefault();
}

public async Task<IEnumerable<Branch>> GetBranches(string projectId, string branchName = "")
{
var request = new RestRequest("projects/{projectId}/repository/branches/{branchName}", Method.GET);
request.AddUrlSegment("projectId", HttpUtility.UrlEncode(projectId));
request.AddUrlSegment("branchName", branchName);

var r = await Client.ExecuteTaskAsync<List<Branch>>(request);
return r.Data;
}

public async Task<Commit> GetCommitBySHA(string projectId, string commitSHA)
{
var r = await GetCommits(projectId, commitSHA : commitSHA);
return r.FirstOrDefault();
}

public async Task<IEnumerable<Commit>> GetCommits(string projectId, string commitSHA = "")
{
var request = new RestRequest("projects/{projectId}/repository/commits/{commitSHA}", Method.GET);
request.AddUrlSegment("projectId", HttpUtility.UrlEncode(projectId));
request.AddUrlSegment("commitSHA", commitSHA);

var r = await Client.ExecuteTaskAsync<List<Commit>>(request);
return r.Data;
}

public async Task<IEnumerable<Job>> GetJobs(string projectId, string pipelineId)
{
var request = new RestRequest("projects/{projectId}/pipelines/{pipelineId}/jobs", Method.GET);
request.AddUrlSegment("projectId", HttpUtility.UrlEncode(projectId));
request.AddUrlSegment("pipelineId", pipelineId);

request.AddQueryParameter("per_page", "10000");//Arbitrary value

var r = await Client.ExecuteTaskAsync<IEnumerable<Job>>(request);
return r.Data;
}

public async Task<IEnumerable<Branch>> SearchForBranchInProject(string projectId, string branchPartialName)
{
var request = new RestRequest("projects/{projectId}/repository/branches?search={branchPartialName}", Method.GET);
request.AddUrlSegment("projectId", HttpUtility.UrlEncode(projectId));
request.AddUrlSegment("branchPartialName", branchPartialName);

request.AddQueryParameter("per_page", "10000");

var r = await Client.ExecuteTaskAsync<List<Branch>>(request);
return r.Data;
}
}
}
Loading