Main Page: Difference between revisions

From Essential
Jump to navigation Jump to search
No edit summary
Line 5: Line 5:


<br>
<br>
==[https://openai.com/ AI tools]==
== AI tools ==
* [https://chrome.google.com/webstore/detail/webchatgpt-chatgpt-with-i/lpfemeioodjbpieminkklglpmhlngfcn Science research]
* [https://chrome.google.com/webstore/detail/webchatgpt-chatgpt-with-i/lpfemeioodjbpieminkklglpmhlngfcn Science research]
* [https://www.midjourney.com Design]
* [https://www.midjourney.com Design]
Line 19: Line 19:
#[https://lmsys.org/blog/2023-05-03-arena Benchmark]
#[https://lmsys.org/blog/2023-05-03-arena Benchmark]


==CLOUD LAB==
== CLOUD LAB ==
I want to share my [[LAB project]].<br>
I want to share my [[LAB project]].<br>
<br>
<br>
[[file:Infocepo.drawio.png]]
[[file:Infocepo.drawio.png]]
==INFRA audit==
 
== INFRA audit ==
I made [[ServerDiff.sh]] script to audit servers.
I made [[ServerDiff.sh]] script to audit servers.
You can track configuration drift.
You can track configuration drift.
You can check if your environments are the same.
You can check if your environments are the same.


==CLOUD migration example==
== CLOUD migration example ==
*1.5 days: infra audit (82 clustered services) ([https://infocepo.com/wiki/index.php/ServerDiff.sh audit own tool])
*1.5 days: infra audit (82 clustered services) ([https://infocepo.com/wiki/index.php/ServerDiff.sh audit own tool])


Line 76: Line 77:
  Total = 15 man-days
  Total = 15 man-days


==CLOUD improvement==
== CLOUD improvement ==
[[File:WebModelDiagram.drawio.png]]
[[File:WebModelDiagram.drawio.png]]
*Formalize your infrastructure as much as possible for more flexibility, low complexity and less technology lock-in.
*Formalize your infrastructure as much as possible for more flexibility, low complexity and less technology lock-in.
Line 99: Line 100:
* [https://wikitech.wikimedia.org/wiki/Wikimedia_infrastructure CLOUD WIKIPEDIA]
* [https://wikitech.wikimedia.org/wiki/Wikimedia_infrastructure CLOUD WIKIPEDIA]


==CLOUD vs HW==
== CLOUD vs HW ==
{| class="wikitable"
{| class="wikitable"
|'''Function'''
|'''Function'''
Line 200: Line 201:
|NFS/SAN
|NFS/SAN
|}
|}
==[https://cloud.google.com/free/docs/aws-azure-gcp-service-comparison CLOUD providers]==
== CLOUD providers ==
* [https://cloud.google.com/free/docs/aws-azure-gcp-service-comparison CLOUD providers]
* [https://cloud.google.com/free/docs/aws-azure-gcp-service-comparison CLOUD providers]
==[https://global-internet-map-2021.telegeography.com/ CLOUD INTERNET NETWORK]==
== CLOUD INTERNET NETWORK ==
* [https://global-internet-map-2021.telegeography.com/ CLOUD INTERNET NETWORK]
* [https://global-internet-map-2021.telegeography.com/ CLOUD INTERNET NETWORK]
==[https://landscape.cncf.io/?fullscreen=yes CLOUD NATIVE]==
== CLOUD NATIVE ==
* [https://landscape.cncf.io/?fullscreen=yes CLOUD NATIVE]
* [https://landscape.cncf.io/?fullscreen=yes CLOUD NATIVE]
==HA COROSYNC+PACEMAKER==
== HA COROSYNC+PACEMAKER ==
[[File:HA-REF.drawio.png]]
[[File:HA-REF.drawio.png]]
===Typical architecture===
=== Typical architecture ===


*2 rooms
*2 rooms
Line 230: Line 231:
For CPU resources, as a rule there are no big requirements. Tip, for time-critical compressions, use PZSTD.
For CPU resources, as a rule there are no big requirements. Tip, for time-critical compressions, use PZSTD.


===Typical service pattern===
=== Typical service pattern ===
*MULTIPATH
*MULTIPATH
*LUN
*LUN
Line 241: Line 242:
*PROCESS (PROCESS resource)
*PROCESS (PROCESS resource)
*LISTENER (LISTENER resource)
*LISTENER (LISTENER resource)
==IT salaries==
== IT salaries ==
*[http://jobsearchtech.about.com/od/educationfortechcareers/tp/HighestCerts.htm Best IT certifications]
*[http://jobsearchtech.about.com/od/educationfortechcareers/tp/HighestCerts.htm Best IT certifications]
*[https://www.silkhom.com/barometre-2021-des-tjm-dans-informatique-digital/ FREELANCE]
*[https://www.silkhom.com/barometre-2021-des-tjm-dans-informatique-digital/ FREELANCE]
*[http://www.journaldunet.com/solutions/emploi-rh/salaire-dans-l-informatique-hays/ IT]
*[http://www.journaldunet.com/solutions/emploi-rh/salaire-dans-l-informatique-hays/ IT]
==[https://openapm.io SRE]==
== SRE ==
* [https://openapm.io SRE]
* [https://openapm.io SRE]
==[https://access.redhat.com/downloads/content/package-browser REDHAT package browser]==
== REDHAT package browser ==
* [https://access.redhat.com/downloads/content/package-browser REDHAT package browser]
* [https://access.redhat.com/downloads/content/package-browser REDHAT package browser]

Revision as of 07:35, 8 May 2023

Ynotopec elementary particles motion interaction science 4f947bd8-3a57-49f5-a5b5-df3128737f22.png

Welcome to my experimental WIKI.

The website covers cloud computing topics like migration, infrastructure, and high availability. It discusses tools like Kubernetes, OpenStack, and AWS, the importance of open-source software, and factors to consider when implementing a cloud infrastructure.


AI tools

  • Next steps :
  1. LANGCHAIN : semantic tool
  2. VICUNA (LLAMA) : OPENSOURCE chat
  • POC :
  1. AUDIO : realtime translate
  2. Coding
  3. AI tools
  4. Benchmark

CLOUD LAB

I want to share my LAB project.

Infocepo.drawio.png

INFRA audit

I made ServerDiff.sh script to audit servers. You can track configuration drift. You can check if your environments are the same.

CLOUD migration example

  • 1.5 days: physical and virtual target CLOUD architecture diagram
  • 1.5 days: physical compliance of 2 CLOUD (6 hypervisors, 6TB memory)
  • 1 days: installation of the 2 CLOUD
  • .5 day: stability check
ACTION RESULT OK/KO
Activate maintenance for n/2-1 nodes or 1 node if 2 nodes. All resources are started.
Un-maintenance all nodes. Power off n/2-1 nodes or 1 node if 2 nodes, different from the previous test. All resources are started.
Power off simultaneous all nodes. Power on simultaneous all nodes. All resources are started.
  • 1.5 days: CLOUD automation study
  • 1.5 days: 6 templates (2 CLOUD, 2 OS, 8 environments, 2 versions)
  • 1 day: migration diagram

Diagram-migration-ORACLE-KVM-v2.drawio.png

  • 1.5 days: process stabilization
  • 1.5 days: CLOUD benchmark vs old INFRA
  • .5 days: calibration of unavailability time per unit migration
  • 5 minutes (effective load): 82 VM (env, os, application_code, 2 IP)
Total = 15 man-days

CLOUD improvement

WebModelDiagram.drawio.png

  • Formalize your infrastructure as much as possible for more flexibility, low complexity and less technology lock-in.
  • Use a name server able to handle the position of your customers like GDNS.
  • Use a minimal instance and use a network load balancer like LVS. Monitor the global load of your instances and add/delete dynamically as needed.
  • Or, many providers have dynamic computing services. Compare the prices. But take care about the technology lock-in.
  • Use a very efficient TLS decoder for the frontend like the HAPROXY decoder.
  • Use very fast http cache like VARNISH.
  • Use a big cache for big files like Apache Traffic Server.
  • ...
  • Use a REVERSE PROXY with TLS decoder like ENVOY for more services compatibility.
  • Use serverless service for standard runtimes like Java, Python and PHP. But beware of certain incompatibilities and a lack of consistency over time.
  • ...
  • Each time you need dynamic computing power think about load balancing or native service from the providers (caution about providers services!)
  • ...
  • Try to use open source STACKs as much as possible.
  • ...
  • Use cache for your databases like MEMCACHED
  • ...
  • For more informations go to CLOUD WIKIPEDIA

CLOUD WIKIPEDIA

CLOUD vs HW

Function KUBERNETES OPENSTACK AWS Bare-metal HPC CRM OVIRT
DEPLOY HELM/ANSIBLE/SH TERRAFORM/ANSIBLE/SH/JUJU TERRAFORM/CLOUDFOUNDATION/ANSIBLE/JUJU ANSIBLE/SH XCAT/CLUSH ANSIBLE/SH ANSIBLE/PYTHON/SH
BOOTSTRAP API/CLI PXE/API/CLI API/CLI PXE/IPMI PXE/IPMI PXE/IPMI PXE/API
Router API/CLI (kube-router) API/CLI (router/subnet) API/CLI (Route table/subnet) LINUX/OVS/external XCAT/external LINUX/external API
Firewall INGRESS/EGRESS/ISTIO API/CLI (Security groups) API/CLI (Security group) LINUX (NFT) LINUX (NFT) LINUX (NFT) API
Vlan/Vxlan many API/CLI (VPC) API/CLI (VPC) OVS/LINUX/external XCAT/external LINUX/external API
Name server coredns dns-nameserver Amazon Route 53 GDNS XCAT LINUX/external API/external
Load balancer kube-proxy/LVS(IPVS) LVS Network Load Balancer LVS SLURM Ldirectord
Storage many SWIFT/CINDER/NOVA S3/EFS/FSX/EBS OPENSTACK SWIFT/XFS/EXT4/RAID10 GPFS SAN NFS/SAN

CLOUD providers

CLOUD INTERNET NETWORK

CLOUD NATIVE

HA COROSYNC+PACEMAKER

HA-REF.drawio.png

Typical architecture

  • 2 rooms
  • 2 power supply
  • 2FC / server (active/active) (SAN)
  • 2*10Gbit/s ethernet / server (active/passive, possible active/active if PXE on native VLAN 0)
  • IPMI VLAN (for the fence)
  • VLAN ADMIN which must be the native VLAN if BOOTSTRAP by PXE (admin, provisioning, heartbeat)
  • USER VLAN (application services)
  • NTP
  • DNS+DHCP+PXE+TFTP+HTTP for auto-provisioning
  • PROXY (for update or otherwise internal REPOSITORY)
  • Choose between 2 or more node clusters.
  • For a 2-node architecture, you need a 2-node configuration on COROSYNC and make sure to configure a 10-second staggered closing for one of the nodes (otherwise, an unstable cluster results).
  • Resources are stateless.

For DB resources it is necessary to provide 4GB per base in general. For CPU resources, as a rule there are no big requirements. Tip, for time-critical compressions, use PZSTD.

Typical service pattern

  • MULTIPATH
  • LUN
  • LVM (LVM resource)
  • FS (FS resource)
  • NFS (FS resource)
  • USER
  • IP (IP resource)
  • DNS name
  • PROCESS (PROCESS resource)
  • LISTENER (LISTENER resource)

IT salaries

SRE

REDHAT package browser