Page Comparison

tis-esr-etl-param-cron-prod

runs mvn

clean install -e

• DELETE:
• replaced by tis-esr-etl which uses service-pipeline

• This is the main job for NDW-ETL
• It is triggered by a push into TIS-NDW-ETL git repo
• It creates the JAR and Docker image by running service-pipeline
• When FINALTAG is set, it deploys esr-etl into dev

• KEEP
• Rename to tis-ndw-etlthe main job for NDW-ETL
• It is triggered by a push into TIS-NDW-ETL git repo
• It creates the JAR and Docker image by running service-pipeline
• When FINALTAG is set, it deploys esr-etl into dev

• KEEP
• Rename to tis-ndw-etl

• This is a cron job in Jenkins
• It can take a parameter: the environment
• It runs ansible/ndw-etl.yml playbook
• It uses UAT inventory

• This is a cron job in Jenkins
• It can take a parameter: the environment
• It runs ansible/ndw-etl.yml playbook
• It uses PROD inventory

• This is a manualjob in Jenkins
• It can take a parameter: the environment
• It runs ansible/ndw-etl.yml playbook
• It uses PROD inventory

• This is a cron job in Jenkins
• It can take a parameter: the environment
• It runs ansible/ndw-etl.yml playbook
• It uses PROD inventory
• DELETED

• 07h00
• daily

tis-ndw-etl-parameterized-uat

• This is a cron job in Jenkins
• It can take a parameter: the environment
• It runs ansible/ndw-etl.yml playbook
• It uses UAT inventory
• DELETED

• 07h00
• daily

• Uses jenkinsfile
• Can run against all our github repositories
• configured to use tis-ndw-etl github repo

• NO IDEA FOR NOW
• Can be used as future implementation ??

disabled because this was a poor attempt to change service-pipeline

DELETED

• This is for manual run only
• One runs it to pass a DEBUG parameter which allows one to attach to port 5005TCP
• It does not create any JAR file or docker image
• It just deploys whatever is in platform.yml into the selected environment
• One can pass the debug parameter to the job, unlike service-env-deploy

• KEEP?
• Duplicate for service-env-deploy 
• But adds DEBUG param

• This is the main job for generic-upload
• It is triggered by a push into TIS-GENERIC-UPLOAD repo
• It creates the JAR and Docker image by running service-pipeline
• When FINALTAG is set, it deploys generic-upload into dev

• This is the main job for tcs
• It is triggered by a push into TIS-TCS git repo
• It creates the JAR and Docker image by running service-pipeline
• When FINALTAG is set, it deploys generic-upload into dev

disabled because this was a poor attempt to change service-pipeline

DELETED

• This is the main job for admins-ui
• It is triggered by a push into TIS-ADMINS-UI github repo
• It creates the JAR and Docker image by running service-pipeline
• When FINALTAG is set, it deploys generic-upload into dev

• User chooses which of our github repositories to run
• Uses jenkinsfile branch of TIS-ADMINS-UI git repo
• In that jenkinsfile branch, there is a file named Jenkinsfile
• That Jenkinsfile file is actually a groovy script with three stages: build the image(BUILD_ID), run tests, then push to registry

• NO IDEA FOR NOW
• See chris-tis-ndw

• runs End to End tests
• active but not a single successfull run
• Uses TIS-ADMINS-UI git repo
• last issue: access to selenium server http://hub-cloud.browserstack.com/wd/hub
• DECISION REQUIRED

• NO IDEA FOR NOW
• This needs to happen so we better fix this somehow?

• This is a clone of the main job for admins-ui
• It used to be triggered by a push into TIS-ADMINS-UI github repo
• It creates the JAR and Docker image by running service-pipeline
• When FINALTAG is set, it deploys generic-upload into dev
• DECISION REQUIRED - last run was on 2018 March 7th

• DISABLE??
• Replace with service-env-deploy

• this is a different implementation of a pipeline
• It uses groovy pipeline directly implemented in the GUI
• It never succeded, last run was on 2018 March 18th
• Last error: java.lang.NoSuchMethodError: No such DSL method 'post' found among steps
• DECISION REQUIRED

• KEEP as POC??
• Can lead to new pipeline

• Uses jenkinsfile
• Can run against all our github repositories
• configured to use TIS-ADMINS-UI github reporepoNone / Automatic

• KEEP

• This is the main job for reference
• It is triggered by a push into TIS-REFERENCE github repo
• It creates the JAR and Docker image by running service-pipeline
• When FINALTAG is set, it deploys generic-upload into dev

• KEEP

• This is the main job for profile
• It is triggered by a push into TIS-PROFILE github repo
• It creates the JAR and Docker image by running service-pipeline
• When FINALTAG is set, it deploys generic-upload into dev

• KEEP
• Rename as tis-profile ??

• This is the main job for profile
• It is triggered by a push into TIS-PROFILE github repo
• It creates the JAR and Docker image by running service-pipeline
• When FINALTAG is set, it deploys generic-upload into dev
• triggers profile
• DUPLICATE ???

• DISABLE??

• This is the main job for assessments
• It is triggered by a push into TIS-ASSESSMENTS github repo
• It creates the JAR and Docker image by running service-pipeline
• When FINALTAG is set, it deploys generic-upload into dev

• KEEP
• rename to tis-assessments ??

• It is triggered by a push into TIS-ASSESSMENT-PROGRAM-MEMBERSHIP github repo
• It creates the JAR and Docker image by running mvn2docker
• When FINALTAG is set, it deploys generic-upload into dev

• It runs the ansible code ansible/assessment-programme-membership-etl.yml

• runs ansible task tasks/audit-logs-backup.yml in dev
• backs audt logs to azure storage container heetislogsbackup, container audit-logs-dev

• midnight
• daily

• KEEP

• runs ansible task tasks/audit-logs-backup.yml in prod
• backs audt logs to azure storage container heetislogsbackup, container audit-logs-prod

• midnight
• daily

• KEEP

• This is the main job for revalidation
• It is triggered by a push into TIS-REVALIDATION github repo
• It creates the JAR and Docker image by running service-pipeline
• When FINALTAG is set, it deploys generic-upload into dev
• triggers ci-test jenkins job 

• KEEP
• rename to tis-revalidation ??

• Runs ansible/tasks/intrepid-reval-etl-all.yml in dev, stage, uidev
• creates docker instance intrepid-dr-etl:{{ versions['intrepid-reval-etl'] }}

• REPLACE by one parameterized job ??

intrepid-reval-etl-all-uat

• Runs ansible/tasks/intrepid-reval-etl-all.yml in UAT
• creates docker instance intrepid-dr-etl:{{ versions['intrepid-reval-etl'] }}

• REPLACE by one parameterized job ??

• Runs ansible/tasks/intrepid-reval-etl-all.yml in prod with --limit 10.170.0.137
• creates docker instance intrepid-dr-etl:{{ versions['intrepid-reval-etl'] }}

• 5h30
• daily

• REPLACE by one parameterized job ??
• prod is to be the default ??

intrepid-reval-etltype-schema-env

• takes ETLTYPE and ENVIRONMENT parameters
• Runs ansible/tasks/intrepid-reval-etl-all.yml --extra-vars="etl_type=${ETLTYPE}"
• creates docker instance intrepid-dr-etl:{{ versions['intrepid-reval-etl'] }}

• KEEP ??
• rename to intrepid-reval-etl-parameterized ??

• runs mvn verify
• triggers release-tag
• last run Apr 19, 2017 1:11:45 PM

• DISABLE ??

• disabled
• uses TIS-REVALIDATION

• disabled
• uses TIS-REVALIDATION

• DISABLE ??

• is triggered by the likes of ci_test
• last run Apr 18, 2017 4:08:28 PM
• takes the following parameters GIT_TAG_URL, GIT_TAG_BRANCH, GIT_TAG_VALUE
• tags the repo GIT_TAG_URL with tag GIT_TAG_VALUE

• FIX it to allow tagging ??

• last run 2017 Feb 27
• pipeline script
• takes a git repo, runs maven on it then runs mvn2docker
• DELETED on 2018-08-22

• DISABLE??

• This is the main job for reference
• It is triggered by a push into TIS-TRAINEE-UI github repo
• It creates the JAR and Docker image by running service-pipeline
• When FINALTAG is set, it deploys generic-upload into dev

• KEEP
• rename to tis-trainee-ui ??

• This is the main job for notifications
• It is triggered by a push into TIS-NOTIFICATIONS github repo
• It creates the JAR and Docker image by running service-pipeline
• When FINALTAG is set, it deploys generic-upload into dev

• KEEP
• rename to tis-notifications ??

• This is the main job for common
• It is triggered by a push into TIS-COMMON github repo
• It creates the JAR and Docker image by running service-pipeline
• When FINALTAG is set, it deploys generic-upload into dev

• KEEP
• rename to tis-common ??

• runs in PROD
• runs ansible code ansible/tasks/intrepid-load.yml
• does not use any git repo, apart from tis-devops
• ansible/tasks/intrepid-load.yml: deploys docker image repository.tis.nhs.uk:5000/hee/csv-diff:{{ versions['intrepid-load'] }}

• KEEP ??

• It is triggered by a push into TIS-INTREPID-DR-MODEL github repo
• runs mvn clean install -e

• KEEP ??

• It is triggered by a push into TIS-INTREPID-DR-TOOLS github repo
• runs mvn clean install -e then mvn2docker

• KEEP ??

• disabled, last run 2017 Nov 9th
• runs ansible playbook ansible/tasks/intrepid-load.yml in PROD
• uses old DEVOPS repo

• DELETE ??

• runs ansible code ansible/tasks/intrepid-anonymizer.yml against localhost
• deploys docker image repository.tis.nhs.uk:5000/hee/anonymizer:{{ versions['intrepid-anonymizer'] }}

• KEEP ??

intrepid-anonymizer-new

• runs ansible/tasks/intrepid-anonymizer-new.yml in STAGE
• never run
• DISABLED on 2018-08-22

• KEEP ??

intrepid-anonymizer-build

Anonymise intrepid data to populate the anonymized mirror

• This is the main job for intrepid-anonymizer
• It is triggered by a push into TIS-DATA-ANONYMIZER github repo
• It creates the JAR and Docker image by running service-pipeline
• When FINALTAG is set, it deploys generic-upload into dev

• KEEP
• rename to tis-data-anonymizer ??

• last run 2017 January 9th
• This is the main job for intrepid-core-model
• It is triggered by a push into TIS-INTREPID-CORE-MODEL github repo
• It creates the JAR and Docker image by running mvn2docker

• KEEP ??

• triggered by intrepid-etl-pipeine
• runs ansible playbook ansible/tasks/intrepid-load.yml in STAGE

• KEEP ??
• merge with another intrepid-load ??

Runs intrepid-extract and load each morning. Intrepid extract: grabs data from intrepid places in into CSV files (db-to-csv). Intrepid remap: changes column names on CSV files (intrepid-remap-col-structure) - we could remove it Intrepid load: grabs CSV data and places it into the mirror (csv-diff) creates a delta file and then uploads to db Intrepid conosolidated etl: points to the mirror and populates the TIS system

• DELETE ??

intrepid-etl-pipeline-new

• pipeline script from SCM
• SCM: TIS-DEVOPS
• script: jenkins/intrepid-etl-prod-new.groovy
• jenkins/intrepid-etl-prod-new.groovy runs ansible/tasks/intrepid-extract-new.yml with intrepid_dr_settings=<dr2|mirror-prod> schema=R1 mirror_settings=mirror-prod
  batch_size=20
  

• 23h05
• daily

• KEEP

• runs ansible/tasks/intrepid-extract-all.yml in PROD
• ansible/tasks/intrepid-extract-all.yml: deploys docker image repository.tis.nhs.uk:5000/hee/intrepid-extract:{{ versions['intrepid-extract'] }}

• KEEP

intrepid-extract-dr2

• disabled, last successful run 2017 Nov 30th
• runs ansible/tasks/intrepid-extract-new.yml on ETL hosts in PROD

• midnight
• daily

• DELETE ??

intrepid-dr-etl-build

• This is the main job for notificationsfor TIS-INTREPID-DR-ETL
• It is triggered by a push into TIS-INTREPID-DR-ETL github repo
• It creates the JAR and Docker image by running service-pipeline
• When FINALTAG is set, it deploys generic-upload into dev

• KEEP

intrepid-compare-stats

• KEEP ??

intrepid-compare-stats-etl

• disabled, last run 2017 Nov 9th
• DELETED on 2018-08-22
• runs ansible/tasks/intrepid-compare-stats.yml in PROD, with DEVOPS_BASE=/datadrive/data/fayaz/master/devops

• DELETE

intrepid-extract-clean

• runs ansible/tasks/intrepid-extract-clean.yml --limit 10.140.0.1 in PROD on ETL 
• runs automatically
• removes old files from /datadrive/data/intrepid
  

• midnight
• daily

• KEEP

intrepid-reval-etl-dev

• runs ansible/tasks/intrepid-reval-etl-all.yml in DEV

• KEEP
• parameterize??

• runs ansible/tasks/intrepid-reval-etl-all.yml in STAGE

• KEEP
• parameterize??

• runs ansible/tasks/intrepid-reval-etl-all.yml in UIDEV

• KEEP
• parameterize??

db-to-csv-parent

• It is triggered by a push into TIS-DB-TO-CSV github repo
• triggers the job intrepid-extract-docker

• KEEP

intrepid-extract-docker

• It is triggered by a push into TIS-DB-TO-CSV github repo
• It can be triggered by db-to-csv-parent
• It runs mvn clean install -e on TIS-DB-TO-CSV/db-to-csv pom.xml file
• It runs ansible/tasks/spring-boot-build.yml in DEV on localhost
• it runs mvn2docker to create an image named intrepid-extract
• When FINALTAG is set, it deploys generic-upload into dev ????

• KEEP
• use the pipeline instead or mvn2docker??

intrepid-extract-upload

• triggered by intrepid-etl-pipeine
• compresses and uploads intrepid databases to azure blob storage ???
• runs ansible/tasks/intrepid-extract-upload.yml in PROD on ETL hosts

• KEEP
• rename to *backup* ??
• automate ??

intrepid-extract-all-etl

• uses export DEVOPS_BASE=/datadrive/data/fayaz/master/devops in PROD
• last run 2017 November 9th
• deploys docker image repository.tis.nhs.uk:5000/hee/intrepid-extract:{{ versions['intrepid-extract'] }}
  
• extracts intrepid databases

• DISABLE ??

intrepid-schema-validation

compares the schema csv exports daily to make sure that the structure hasn't changed.

• KEEP
• REPLACE WITH new pipeline ??

intrepid-etl-pipeline-stage

Runs the second part of the pipeline for intrepid-etl on the mirror stage
Intrepid data anonymizer - takes data from ssh hee-build-azure -> /datadrive/data/intrepid/prod/ (full+delta) anonymises it then places it into /datadrive/data/intrepid/stage/ (full+delta)
Intepid Load Mirror -> takes data from /datadrive/data/intrepid/stage/ , creates a delta file then loads it into mirror stage
Intrepid revalidation stage -> the reval etl on stage

• KEEP??
• Replace with new pipeline ??

intrepid-etl-pipeline-stage-new

Runs the second part of the pipeline for intrepid-etl on the mirror stage
Intrepid data anonymizer - takes data from ssh hee-build-azure -> /datadrive/data/intrepid/prod/ (full+delta) anonymises it then places it into /datadrive/data/intrepid/stage/ (full+delta)
Intepid Load Mirror -> takes data from /datadrive/data/intrepid/stage/ , creates a delta file then loads it into mirror stage
Intrepid revalidation stage -> the reval etl on stage

• KEEP ??
• Replace with new pipeline ??

intrepid-extract-upload-etl

compresses and uploads intrepid databases in azure

• DELETE ??

intrepid-remapper-structure

Remaps the Intrepid DR structure

• KEEP ??

intrepid-extract-consolidated

• runs ansible/tasks/intrepid-extract-consolidated.yml in PROD
• ansible/tasks/intrepid-extract-consolidated.yml does not exist
• last successfull run: 2017-07-23

• DELETE ??

intrepid-remapper-structure-etl

• disabled, last run 2017 Nov 9th
• DELETED on 2018-08-22
• runs ansible/tasks/intrepid-remapper-structure.yml with DEVOPS_BASE=/datadrive/data/fayaz/master/devops

• DELETE ??

intrepid-reval-etltype-schema-env

• runs ansible/tasks/intrepid-reval-etl-all.yml --extra-vars="etl_type=${ETLTYPE}"
• ETLTYPE= <arcp|connectionDiscrepancies|contactDetails|placements|revalidation|selfDeclaration|traineeCard>
• ansible/tasks/intrepid-reval-etl-all.yml deploys repository.tis.nhs.uk:5000/hee/intrepid-dr-etl:{{ versions['intrepid-reval-etl'] }}
  

• KEEP

test-intrepid-connection

• disabled, last run 2017 March 13th
• DELETED on 2018-08-22
• runs /opt/mssql-tools/bin/sqlcmdsqlcmd  -S "TCP:212.250.207.206,4589" -U "TRANSFORM SYSTEM" -P "ed0hUbLT" -d "Intrepid.DataRepository" -Q "quit"

• DELETE??

consolidated-dr-etl-all-parameterised-dev

• DISABLED
• pipeline script from SCM
• SCM: TIS-DEVOPS
• script: jenkins/consolidated-dr-etl.groovy
• jenkins/consolidated-dr-etl.groovy: runs ansible/tasks/consolidated-dr-etl-old.yml or ansible/tasks/consolidated-dr-etl-all.yml
  
  

• DELETE ??

• DELETE

• collects lots of parameters
• runs jenkins/consolidated-dr-etl.groovy
• jenkins/consolidated-dr-etl.groovy: runs ansible/tasks/consolidated-dr-etl-old.yml or ansible/tasks/consolidated-dr-etl-all.yml
• by default, ENVIRONMENT=prod

• KEEP

consolidated-dr-etl-all-parameterised-prod

• collects lots of parameters
• runs jenkins/consolidated-dr-etl.groovy
• jenkins/consolidated-dr-etl.groovy: runs ansible/tasks/consolidated-dr-etl-old.yml or ansible/tasks/consolidated-dr-etl-all.yml
• by default, ENVIRONMENT=prod

• 05h30
• daily

• KEEP

consolidated-dr-etl

• This is the main job for TIS-CONSOLIDATE-DR-ETL
• It is triggered by a push into TIS-CONSOLIDATED-DR-ETL github repo
• It creates the JAR and Docker image by running mvn2docker

• KEEP
• use the pipeline instead of mvn2docker ??

consolidated-dr-etl-old

• This is the main job for intrepid-core-model
• It is triggered by a push into TIS-CONSOLIDATED-DR-ETL-OLD github repo
• It creates the JAR and Docker image by running mvn2docker

• KEEP
• use the pipeline instead of mvn2docker ??

consolidated-dr-etl-all-dev

• runs ansible/tasks/consolidated-dr-etl-all.yml in DEV

• KEEP

consolidated-dr-etl-old-dev

• runs ansible/tasks/consolidated-dr-etl-old.yml  --limit 10.150.0.137 in DEV 

• KEEP

consolidated-dr-etl-all-prod

• runs ansible/tasks/consolidated-dr-etl-all.yml  in PROD
• deploys repository.tis.nhs.uk:5000/hee/consolidated-dr-etl:{{ versions['consolidated-dr-etl'] }}
  

• KEEP

consolidated-dr-etl-old-prod

NOTE THIS IS THE OLD CONSOLIDATED DR JOB

• KEEP ??

consolidated-dr-etl-all-stage

• runs ansible/tasks/consolidated-dr-etl-all.yml  in STAGE
• ansible/tasks/consolidated-dr-etl-all.yml: deploys repository.tis.nhs.uk:5000/hee/consolidated-dr-etl:{{ versions['consolidated-dr-etl'] }}
  

• KEEP

consolidated-dr-etl-all-uidev

• runs ansible/tasks/consolidated-dr-etl-all.yml in UIDEV
• ansible/tasks/consolidated-dr-etl-all.yml: deploys repository.tis.nhs.uk:5000/hee/consolidated-dr-etl:{{ versions['consolidated-dr-etl'] }}

• KEEP

consolidated-dr-etl-old-stage

NOTE THIS IS THE OLD CONSOLIDATED DR JOB 
https://github.com/Health-Education-England/TIS-CONSOLIDATED-DR-ETL-OLD 

• KEEP ??

• KEEP ??

• ANY environment
• runs ansible/tasks/consolidated-dr-etl-all.yml --extra-vars="schema=${SCHEMA}"
• deploys repository.tis.nhs.uk:5000/hee/consolidated-dr-etl:{{ versions['consolidated-dr-etl'] }}
  

• KEEP

• DISABLED
• uses /datadrive/data/fayaz/master/devops to run ansible/tasks/consolidated-dr-etl-all.yml in PROD

• DELETE ??

ansible-cmd

• runs /usr/local/bin/ansible -i /home/jenkins/data/devops/ansible/inventory/all -m setup --tree /datadrive/data/ansible-cmdb/ all
• puts the result in /var/www/html/reports/ansible-cmdb

• 7h15
• daily

• KEEP

bdd_tests_staging

• DELETE

• last run on Nov 29, 2016 5:57:22 PM
• uses TIS-REVALIDATION repo
• runs "npm install && gulp bs" in DEV ???
• DISABLED on 2018-08-22

• DELETE

• last run Jun 19, 2017 4:02 PM
• uses TIS-REVALIDATION repo
• runs the  ci-stage-pipeline pipeline

• DISABLE ??

ci-stage-pipeline

• last run Jun 19, 2017 4:02:41 PM
• runs a pipeline script which runs a "mvn verify" on REVALIDATION, CONCERNS and NOTIFICATIONS

• DISABLE??

ci-staging

• last run Feb 20, 2017 4:46:10 PM
• watches site-stage
• triggers intrepid-reval-etl-stage

• DISABLE ??

ci_test

• uses TIS-REVALIDATION repo

• 
  

  Code Block
  export TIS_ENV=development export CHROME_DRIVER_HOME=/usr/bin/ cd bdd mvn verify

  
  

• triggers release-tag jenkins job

• DISABLE ??

ci-tests-concerns

• last run Feb 21, 2017 2:42:59 PM
• uses TIS-CONCERNS git repo
• runs "mvn verify"

• DISABLE ??

ci_tests_parallel

• DISABLED
• last run Sep 27, 2016 8:48:22 AM
• uses TIS-REVALIDATIONS git repo
• watches jenkins job revalidation-dev-deploy which does not exist

• DELETE ??

ci_tests_tagging

• DISABLED
• Never run
• uses TIS-REVALIDATIONS git repo
• runs selenium
• triggers non existing git-tag jenkins job

• DELETE ??

csv-diff

• uses db-to-csv git repo
• runs ansible/tasks/spring-boot-build.yml ansible playbook with ARTIFACT_ID="intrepid-load"
• ansible/tasks/spring-boot-build.yml runs mvn2docker
  
• last run Nov 15, 2017 10:15:45 AM

• KEEP ??

csv-to-db

• uses csv-to-db git repo
• runs ansible/tasks/spring-boot-build.yml ansible playbook with ARTIFACT_ID="intrepid-load"
• ansible/tasks/spring-boot-build.yml runs mvn2docker
  
• last run Nov 15, 2017 10:15:45 AM

• KEEP ??

curator-docker

• last run Apr 18, 2017 10:06:07 AM
• Rebuilds the Curator docker Image
• uses TIS-CURATOR git repo
• runs /home/jenkins/data/devops/bin/docker-build-push.sh which runs docker build|push|tag|rmi
  

• KEEP
• rename to tis-curator ??

db-to-csv

• last run Nov 15, 2017 10:15:45 AM
• uses db-to-csv git repo
• runs ansible/tasks/spring-boot-build.yml ansible playbook with ARTIFACT_ID="intrepid-load"
• ansible/tasks/spring-boot-build.yml runs mvn2docker

• KEEP ??

db-to-csv-archive

• DISABLED
• uses db-to-csv git repo
• last run Mar 16, 2017 12:54:04 PM

• DELETE ??

db-to-csv-parent

• last run Nov 15, 2017 10:15:45 AM
• uses TIS-DB-TO-CSV git repo
• triggers intrepid-extract-docker jenkins job

• KEEP ??

db-to-csv-test-archive

• DISABLED
• last run Mar 20, 2017 1:51:27 PM
• uses db-to-csv git repo
• runs /tmp/fayaz/TIS-DEVOPS/ansible/tasks/spring-boot-build.yml

• DELETE ??

dev-api-load-test

• never run
• uses TIS-JMETER git repo
• watches ci-staging
• runs "./jmeter.sh -n -t load.jmx -l testresults.jtl"

• DISABLE ??

devops

• uses TIS-DEVOPS git repo
• runs python "${WORKSPACE}/ansible/inventory/tests.py"
• and then runs rsync -av --delete "${WORKSPACE}/" --exclude ".git" /home/jenkins/data/devops/
• and rsync -ave 'ssh' --delete "${WORKSPACE}/" --exclude ".git" $(/home/jenkins/data/devops/bin/ip etl-stage):/home/heetis/data/devops/

• KEEP
• rename to tis-devops ??

docker-alpine-java

• DISABLED
• last run Dec 16, 2017 12:41:00 AM
• uses docker-alpine-java git repo
• runs /home/jenkins/data/devops/bin/docker-build-push.sh
• /home/jenkins/data/devops/bin/docker-build-push.sh runs docker build|push|tag|rmi

• midnight
• daily

• DELETE ??

docker-clean-build

• 23h00
• friday

• DELETE ??

docker-clean-dev

Runs regularly to tidy up stray docker artifacts such as "<none>" images, old stopped containers and anonymous volumes

• uses DEV ansible inventory
• runs ansible/tasks/docker-clean.yml playbook
• ansible/tasks/docker-clean.yml: runs docker system prune -af

• 23h00
• friday

• KEEP

docker-clean-prod

• 
  

  Code Block
  Runs regularly to tidy up stray docker artifacts such as "<none>" images, old stopped containers and anonymous volumes

  
  

• uses PROD ansible inventory
• runs ansible/tasks/docker-clean.yml playbook
• ansible/tasks/docker-clean.yml: runs docker system prune -af

• 23h00
• daily

• KEEP

docker-clean-stage

• 
  

  Code Block
  Runs regularly to tidy up stray docker artifacts such as "<none>" images, old stopped containers and anonymous volumes

  • uses STAGE ansible inventory
  • runs ansible/tasks/docker-clean.yml playbook
  • ansible/tasks/docker-clean.yml: runs docker system prune -af

• 23h00
• daily

• KEEP

docker-clean-uidev

Runs regularly to tidy up stray docker artifacts such as "<none>" images, old stopped containers and anonymous volumes

• 23h00
• daily

• KEEP

docker-freeze-dev

Freezes docker images on a given platform

• KEEP
• paramaterize??

docker-freeze-prod

• KEEP
• parameterize??

docker-registryui-cert-renew

• 6h06
• sunday

• DELETE??

docker-registry-ui-dev-deploy

redeploys the docker-registry-ui docker container onto the docker-registry instance on port 5001

• KEEP

elasticsearch-snapshot-dev

• runs ansible/tasks/es_snapshot.yml ansible playbook
• uses DEV inventory

• 23h00
• daily

• KEEP

elasticsearch-snapshot-prod

• runs ansible/tasks/es_snapshot.yml ansible playbook
• uses PROD inventory

• 23h00
• daily

• KEEP

elasticsearch-snapshot-stage

• runs ansible/tasks/es_snapshot.yml ansible playbook
• uses STAGE inventory

• 23h00
• daily

• KEEP

final-services

• last run Jun 1, 2017 2:52:59 PM

• 
  

  Code Block
  List of jobs/services to apply final tag. It should be comma and space separated and should be one or more from [common, concerns, gmc-connect, gmc-sync, inspectlet, new-core, notifications, profile, reference, revalidation, trainee-id, tcs]

  
  

• KEEP ??

gmc_api_integration_tests

• last run Oct 18, 2016 3:49:40 PM
• uses REVALIDATION git repo
• runs commands similar to ```curl --trace trace.txt -i -XPOST -H 'Content-Type: text/xml' -H 'SOAPAction: https://webcache.gmc-uk.org/GMCWebServices/GetDoctorsForDB' https://webcache.gmc-uk.org/GMCWebServicesExtTest/WebService.asmx ```

• KEEP ??
• rename ??

gmc-connect

• last run Aug 18, 2017 1:40:58 PM
• uses TIS-GMC-CONNECT git repo
• triggers service-pipeline  

• KEEP ??

gmc-sync

• last run Sep 25, 2017 9:21:30 PM
• uses TIS-GMC-SYNC git repo
• triggers service-pipeline  

• KEEP

gmc-sync-dev

Runs the GMC ETL process on DEV

• KEEP ?

gmcsync-etltype-schema-env

• KEEP

gmc-sync-prod

Runs the GMC ETL process on prod

• 2h30
• monday to friday

• KEEP

gmc-sync-stage

• runs  ansible/tasks/gmcsync.yml ansible playbook
• uses STAGE inventory

• KEEP
• parameterize??

gmc-sync-uat

• DISABLED
• runs --limit=10.160.0.141 ansible/tasks/gmcsync.yml

• 2h30
• monday to friday

• KEEP
• parameterize??

hee-alpine-java-docker

• watches docker-alpine-java, hee-alpine-jre
• runs ```IMAGE=hee-alpine-java WORKSPACE=~/data/devops/docker/images/hee-alpine-java ~/data/devops/bin/docker-build-push.sh```
• last run Jan 16, 2018 2:00:14 AM

• DELETE ??

hee-alpine-jre

• NEVER run successfully
• runs ```IMAGE=hee-alpine-jdk WORKSPACE=~/data/devops/docker/images/hee-alpine-jdk ~/data/devops/bin/docker-build-push.sh```

• midnight
• daily

• DELETE ??

hee-alpine-nginx-docker

Builds a docker image based on smebberson/alpine-nginx.

• midnight
• daily

• KEEP??

hee-alpine-npm-docker

Builds a docker image based on mhart/alpine-node.

• midnight
• daily

• KEEP??

hee-grant-access

Grant access to hee components(docker registry ui, maven repository, ssh).

• runs ansible/tasks/hee-access.yml ansible playbook
• uses ALL inventory

• KEEP

hee-revoke-access

• runs ansible/tasks/hee-access.yml
• uses ALL inventory

• 
  

  Code Block
  Revoke access to hee components(docker registry ui, maven repository, ssh)

  
  

• KEEP

iam

• KEEP??

inspectlet

• last run Mar 17, 2017 2:40:29 PM
• uses TIS-INSPECTLET git repo
• triggers service-pipeline  

• DISABLE ??

java-library-base

Sample project to clone Java Library projects from.

• DISABLE ??

jenkins-cert-renew

• DISABLED
• runs ansible/tasks/jenkins-cert-renew.yml
• uses BUILD repo

• 6h06
• saturdays

• DELETE ??

jenkins-dsl-seed

• runs DSL code
• buildMonitorView('Production ETLs')
• buildMonitorView('Microservice builds')

• KEEP ??

jmxmp-agent

• last run Dec 11, 2017 11:45:26 AM
• uses TIS-JMXMP-AGENT git repo
• triggers mvn2docker jenkins job

• KEEP
• replace mvn2docker with pipeline ??

• uses TIS-KEYCLOAK-CLIENT git repo
• triggers mvn2docker jenkins job
• tag the repo

• KEEP
• replace mvn2docker with pipeline ??

keycloak-db-sync

• NEVER run
• uses TIS-KEYCLOAK-DB-SYNC git repo
• triggers mvn2docker jenkins job

• KEEP ??
• replace mvn2docker with pipeline ??

keycloak-docker

Rebuilds the customised Keycloak docker image.

• uses TIS-KEYCLOAK-THEME git repo
• runs ${WORKSPACE}/build.sh
• and runs ansible/keycloak.yml with DEV inventory

• KEEP
• rename to tis-keycloak-theme??

lms

• last run Dec 17, 2017 9:19:31 AM
• uses TIS-MICROSERVICE-DESIGN git repo
• runs the following

• 
  

  Code Block
  mvn -DskipTests clean package python ./bin/merge.py rsync -av --delete $WORKSPACE/swagger-ui/ /var/www/html/doc/lms/swagger-ui/ rsync -av --delete $WORKSPACE/static/ /var/www/html/doc/lms/static/

  
  

• KEEP??

london-dr-model

• KEEP??

managed-disks-snapshots

• runs ansible/tasks/managed-disks-snapshot.yml
• uses BUILD inventory
• ansible/tasks/managed-disks-snapshot.yml runs ```az snapshot create```
  

• 21h00
• daily

• KEEP
• Failed since May 5th

maven-poms

• NEVER run
• uses TIS-MAVEN-POMS git repo
• runs ```mvn clean install``

• DISABLE ??

monitoring-health-prod

• runs ansible/tasks/monitoring-health.yml
• uses PROD inventory
• checks https://{{ server_name }}/alertmanager/api/v1/status
  

• every 5 minutes

• KEEP

mvn2docker

• runs /home/jenkins/data/devops/bin/mvn2docker

• KEEP
• eventually DISABLE ??

mysql-backup-dev

Create mysql backup on dev environment mysql instance

• midnight
• daily

• KEEP

mysql-backup-prod

• runs ansible/tasks/mysql-backup.yml
• uses PROD inventory

• midnight
• daily

• KEEP

mysql-backup-stage

• runs ansible/tasks/mysql-backup.yml
• uses PROD inventory

• midnight
• daily

• KEEP

mysql-dev-deploy

• last run Oct 6, 2016 1:20:09 PM
• runs ansible/mysql.yml which does not exit
• uses AZURE inventory which does not exist

• DELETE??

n3-bridge

• last successful run Oct 8, 2017 4:58:54 PM
• runs ansible/n3-bridge.yml
• uses SHARED inventory

• KEEP
• test ??

node2docker

Builds a docker container from node front-end/package.json

• runs /home/jenkins/data/devops/bin/node2docker

• KEEP
• eventually DISABLE??

oriel-etl

Sample project to clone Java Library projects from.

• KEEP??

oriel-etl-dev

• DISABLE??

owasp-dependency-check

• last run Oct 30, 2016 4:53:38 PM
• uses TIS-REVALIDATION git repo

• runs

  Code Block
  npm install bower install mvn clean install cd bdd npm install cd ..

  
  

• KEEP??

owasp_zap

• last run Nov 17, 2016 12:00:09 PM
• uses TIS-REVALIDATION git repo
• install ZAProxy_2.5.0 with https://dev-api.transformcloud.net/revalidation/ as a target
• uses workspace /home/jenkins/data/jenkins/workspace/owasp_zap

• DISABLE ??

person-owner-post-view-rebuild

Runs the person-owner and post-view rebuild nightly.

• runs ansible/tasks/person-owner-rebuild.yml
• uses PROD inventory

• 7h30
• daily

• KEEP

piwik-docker

• last run Dec 8, 2016 4:44:29 PM
• uses docker-piwik git repo
• runs bin/docker-build-push.sh
• bin/docker-build-push.sh which is a wrapper around docker build|push|tag

• DISABLE ??

postgresql-backup-build

• midnight
• daily

• DELETE ??

registry-clone

• DISABLE ??

release-dev

• run ONCE only Nov 14, 2017 10:54:23 AM

• 
  

  Code Block
  TIS-Release deployment in blue-green mode

  
  

• uses /datadrive/data/fayaz/es/TIS-DEVOPS
• runs various ansible playbooks

• DISABLE ??
• duplicate of release-dev-pipeline ??

release-dev-pipeline

• runs a pipeline script which runs ansible/api-gateway.yml and ansible/releases/<RELEASENB>/main.yml

release-prod

• DISABLED
• last run Nov 13, 2017 11:15:51 AM
• runs ansible/releases/<RELEASENB>/main.yml
• uses PROD inventory

• DELETE ??

release-prod-pipeline

• runs a pipeline script which runs ansible/api-gateway.yml and ansible/releases/<RELEASENB>/main.yml
• uses PROD inventory

• KEEP

release-sandbox-pipeline

• runs a pipeline script which runs ansible/api-gateway.yml and ansible/releases/<RELEASENB>/main.yml
• uses SANDBOX inventory

• KEEP ??

release-stage-pipeline

• runs a pipeline script which runs ansible/api-gateway.yml and ansible/releases/<RELEASENB>/main.yml
• uses STAGE inventory

• KEEP

release-tag

• runs jenkins git publisher plugin to tag the repo

• KEEP ??

release-uat-pipeline

• runs ansible/releases/${RELEASE_NUMBER}/main.yml
• uses UAT inventory

• KEEP

Sandbox-Down

• runs a bunch of ```az vm stop ``` commands

• 7pm
• Monday to friday

• KEEP

Sandbox-up

• uns a bunch of ```az vm start ``` commands

• 6h30
• Monday to friday

• KEEP

service-docker

• never run
• takes a repo name and runs bin/docker-build-push.sh
• bin/docker-build-push.sh is a warpper around "docker pull|push|tag"
• triggers service-env-deploy 

• DISABLE ??

service-env-deploy

• runs ansible/${SERVICE}.yml
• uses ansible/inventory/${ENVIRONMENT} inventory

• KEEP

service-final-tag

• runs a shell script defined in jenkins
• shell script is a wrapper around "docker pull|tag|push|rmi"
• last run Mar 6, 2017 8:24:10 AM

• DISABLE ??

service-pipeline-adminsui

• never run
• testing a new way of using the service-pipeline
• To be DELETED
• DELETED

• DELETE

service-pipeline-bkp

• DISABLED
• pipeline script
• copy and paste of service-pipeline.groovy ????

• DELETE ??

service-pipeline-ndw-etl

• DISABLED
• uses a custom pipeline script jenkins/service-pipeline-ndw-etl.groovy
• To be DELETED
• DELETED

• DELETE

service-pipeline-scm

• runs a pipeline script for SCM Jenkinsfile
• uses TIS-JENKINS git repo

• KEEP
• will replace service-pipeline??

service-registry

• runs ansible/tasks/host-registry.yml
• uses DEV, STAGE and PROD inventory

• every 5 minutes

• KEEP

service-test

• uses TIS-INSPECTLET git repo
• triggers service-pipeline 

• KEEP ??

site-dev

• midnight
• daily

• KEEP
• Failed for 2 months

site-dev3

• last run Nov 21, 2016 4:00:51 PM
• runs ansible playbook ansible/site.yml
• uses DEV3 ansible inventory

• DISABLE

site-prod

• last run Oct 5, 2017 7:11:53 AM
• runs ansible playbook ansible/site.yml
• uses PROD ansible inventory

• KEEP
• test??

site-stage

• runs ansible playbook ansible/site.yml
• uses STAGE ansible inventory

• KEEP

site-uidev

• runs ansible/platform-services.yml and ansible/uidevservices.yml
• uses UIDEV ansible inventory

• KEEP
• Test??

sonarqube

• last run Oct 5, 2016 11:01:42 AM

• runs

  Code Block
  docker stop sonarqube || true docker rm sonarqube || true docker run -d --name sonarqube -p 9000:9000 -p 9092:9092 sonarqube

  
  

• DISABLE ??

ssh

• runs ansible/tasks/ssh.yml
• uses ansible inventory BASTION, DEV, UIDEV, STAGE, PROD

• KEEP

staging-api-load-test

• last run Dec 14, 2016 4:17:24 PM
• uses  TIS-JMETER git repo
• watches ci-staging jenkins job

• runs

  Code Block
  cd bin ./jmeter.sh -n -t load.jmx -l testresults.jtl

  
  

• publishes bin/testresults.jtl;bin/jmeter.log

• DISABLE ??

terraform-etl

• runs a pipeline script
• pipeline runs ansible/tasks/terraform-etl.yml with different tags

• DISABLE ??

test-intrepid-connection

• every 15 minutes
• between 0h00 and 8h00

• DELETE ??

test-service-pipeline

• last run Oct 19, 2017 6:54:58 PM
• runs a pipeline script
• pipeline script very similar to service-pipeline.groovy

• DISABLE ??

tis-core-model

• run twice, last was May 26, 2017 2:15:36 PM
• uses TIS-CORE-MODEL git repo

• runs mvn

  Code Block
  clean install -e

  
  

• DISABLE ??

tis-core-services

• DISABLE ??

tis-shared-modules

• 
  

  Code Block
  common modules to be shared across different services.

  
  

• uses TIS-SHARED-MODULES git repo

• runs mvn

  Code Block
  clean install -e

  
  

• then archives the result in **/target/*.jar

• KEEP

uat-sync-from-prod

• 6am
• daily

• KEEP
• Rename ??

ubuntu-security-updates

• DELETE ??

ui-pipeline

• run once on Oct 19, 2017 1:08:22 PM
• runs a pipeline script
• pipeline script similar to service-pipeline.groovy

• DISABLE ??