537 Commits

Author SHA1 Message Date
sai battina
505a8faccf Add env variables for necessary roles.
This PS also updates airship config site details to be passed as a variable

Change-Id: I299c2cd932a2d44486907a2435306348ea3e4d00
2020-04-16 20:42:31 +00:00
Ian Howell
e9f8ac3ac3 Add APIExtensions ClientSet to airship Client
This change adds the APIExtension ClientSet to airship's client wrapper,
allowing interactions with CRDs.

Change-Id: Ic156551fde132754915cdb957dd3d82221744cc3
2020-04-15 12:22:03 -05:00
Zuul
1ec7a24b54 Merge "Uplift go-git" 2020-04-15 13:08:33 +00:00
Alexander Hughes
c70f4a2742 Refactor of codebase addressing IDE errors
Goland is complaining of a number of bad practices, and unused code.
This patchset addresses some of those complaints, such as:
- struct initialization without field names
- error string should not be capitalized or end with punctuation
- name starts with package name
- duplicate yaml keys
- snake case
- redundant parentheses

For the http.Transport redundant parentheses, the lint target fails
without the use of //nolint:errcheck so it has been added as
(currently) there is no potential for there to be an error returned
see [0] for example.

[0] https://golang.org/src/net/http/transport.go#L42

Change-Id: Ib1b20639c9b6e9be097ef1f158ecd7472f578488
Signed-off-by: Alexander Hughes <Alexander.Hughes@pm.me>
2020-04-14 15:55:04 -04:00
Ian Howell
7c6de22354 Uplift go-git
The go-git module (and helper modules)  recently moved from
`gopkg.in/src-d/go-git` to `github.com/go-git/go-git` [0]. This commit
switches to the new module, as well as uplifts from v4 to v5.

[0] https://github.com/src-d/go-git/issues/1295#issuecomment-592965250

Change-Id: I2e73ba7bb1220a3c75c0d81aa5ab0f1f13479161
2020-04-14 09:07:28 -05:00
Zuul
8438935cfe Merge "Update Kustomize integration to api/v0.3.1" 2020-04-14 11:36:33 +00:00
Zuul
f520dc1190 Merge "Change Label and Annotate to accept map of string" 2020-04-13 19:07:47 +00:00
Zuul
9eb5449b5c Merge "Switching local gating scripts to Python3 to comply with Zuul" 2020-04-13 18:24:24 +00:00
Zuul
e7380b3ba6 Merge "Remove getters/setters of AirshipCTLSettings" 2020-04-13 16:15:02 +00:00
Matt McEuen
e52ed80363 Update Kustomize integration to api/v0.3.1
This updates the Kustomize dependency for airshipctl to
api/v0.3.1, which is the oldest version which will support the
Replacement Transformer plugin.

Some changes were needed to accomodate the fact that various apis
that airshipctl was relying on were moved to kustomize-insternal
packages, namely:
- Integrated with the krusty.Kustomizer to drive kustomization
- Removed the custom plugin loader which leveraged the Unknown type
- Worked around NoFieldError becoming private, inc. removing a test

As a follow on we'll need to re-integrate plugin functionality somehow.

Also, in this release Kustomize has implemented support for the
"config.kubernetes.io/local-config" annotation, which we'd planned
to use to to avoid deploying some documents to the Kubernetes API.
It turns out the semantics are different than we anticipated;
Kustomize also fails to return these docs via document *selection*.
Therefore, this change reverts to an earlier approach which uses
a custom airshipit.org/deploy-k8s label.

Change-Id: I7022e12464ea7b6a3ca8609f99f3699bf8da0edd
2020-04-13 10:06:41 -05:00
Zuul
121adcddc8 Merge "Fixed TODO about error definitions" 2020-04-13 13:34:51 +00:00
Alexey Odinokov
1505353333 Switching local gating scripts to Python3 to comply with Zuul
The initial version of gating playbook contained ability
to run them either on Python2 or Python3 ansible.
But it causes a lot of issues e.g. a need to install[1] both
versions of libs to support both possible ways to run,
discrepancy between local and Zuul run[2]
Zuul testing is done with ansible running on Python3 [3].
Meanwhile ppa repo for ansible contains ansible pachage
that depends only from python2. There is no package for
Python3 and we used only Python2-ansyble.
The fact that Zuul runs it with different configuration
means that we'll keep facing the issues with dependencies
in future, since Gating doesn't test it.
This patch changes makes local scripts run as Zuul.

PS
The script partially takes care of reverting intallation
from its previous runs. To cleanup completely it's possible
to run [4] in case these pacakges are not used by anyone else

[1]
https://opendev.org/airship/airshipctl/src/branch/master/roles/libvirt-install/tasks/main.yaml#L38
[2]
https://review.opendev.org/#/c/717841/
https://review.opendev.org/#/c/718590/
[3]
https://zuul.opendev.org/t/openstack/build/4096513604a84cd2b2f2578fc3f98660/log/zuul-info/host-info.primary.yaml#498
[4]
if dpkg -l | grep "ansible.*ppa~bionic"; then
      sudo apt -y remove ansible
      sudo apt -y autoremove
      sudo add-apt-repository -r -y ppa:ansible/ansible
      sudo apt-get -y update
fi
+ possibly
sudo apt remove virtinst python-libvirt python-docker python-requests python-lxml python-libvirt

Change-Id: Ia1ac3aa4128f6c4797dea1709ccb8516ae342a66
2020-04-11 06:53:40 +00:00
Zuul
788dc35b0a Merge "Add copyright for missing files" 2020-04-10 17:51:51 +00:00
Kostiantyn Kalynovskyi
b485f00c58 Change Label and Annotate to accept map of string
The methods are useful when incoming data are not key value pairs,
rather than iterating two times, one time when we call Annotate()
or Label() over the map, and then iterate again inside the function,
it would be better if we could skip iterating the first time, and
let everything to be handled by Annotate or Label functions

Relates-To: #162

Change-Id: Ia9d989a0534d15e4024317c4606a187ac021b751
2020-04-09 16:34:05 -05:00
Zuul
992efae971 Merge "Fixed local gate scripts" 2020-04-09 13:39:55 +00:00
Yasin, Siraj (SY495P)
c25d223c7b Add copyright for missing files
* added license templates for go, bash & yaml in tools dir
* added a script that will add license information for all
    missing files. Type:  go, yaml, yml, sh
* skip adding license for all files within testdata
* Syntax:
   > ./tools/add_license.sh

* Skip license for manifests folder
* Added one extra line after licene for yaml files
* Added License after Hashbang for bash.
* Add an extra line after hashbang and before license
* Updated the go template to use multiline comments

New Files:
  1. tools/add_license.sh
  2. tools/license_go.txt
  3. tools/license_yaml.txt
  4. tools/license_bash.txt

Change-Id: Ia4da5b261e7cd518d446896b72c810421877472a
Realtes-To:#147
2020-04-09 08:35:59 -05:00
Zuul
780c6b8459 Merge "Update Kustomize to v3.3.1" 2020-04-09 10:07:30 +00:00
Zuul
9ad4db66f4 Merge "Fix for Lint warnings" 2020-04-09 10:01:26 +00:00
Stanislav Egorov
9e0b8b2a96 Fixed local gate scripts
Change-Id: I3fc0c597f1b4f050b40905b5415069c5d0b8101f
2020-04-08 18:31:30 -07:00
Zuul
a306429749 Merge "Create documentation for new developers" 2020-04-08 15:58:57 +00:00
Noor Muhammad Malik
e650468379 Create documentation for new developers
This patch updates the develop documentation in docs/source/developers.md.
Adds details on environment setup and uses some of the scripts available
in tools/gate directory.

Change-Id: I76d426746f668f89ab4f9e43ac00f75f47735702
Relates-To: #72
Closes: #72
2020-04-08 19:22:27 +05:00
Ian Howell
5efc26d9f0 Remove getters/setters of AirshipCTLSettings
This change removes the getters and setters for the following attributes
of pkg/environment.AirshipCTLSettings:

* airshipConfigPath
* kubeConfigPath
* config

The above attributes have been exported to account for the removal of
these methods.

Change-Id: I4e7d38bf6771f7521179a2fe2990475b936b5d95
2020-04-08 09:00:32 -05:00
Yasin, Siraj (SY495P)
f696ed5f11 Fix for Lint warnings
* This commit handles only warnings of mentioned type
  warning:
    comment on exported function <funcname> should be
      of the form "<funcname> ..."
* In this case, there were comments for the exported
  function/constant/var for which warnings raised,
  but not in expected format.
* So this fix is mostly formating the comment to avoid
  lint warnings.

  Observations:
    * comment did not had function name
    * function name in comment was immediately follwed by comma
        => function followed by space and any description is expected
    * function was not mentioned in the first line of the comment
        => when there is multiline comment, first line should start with
           function name

Change-Id: Ife97104ebc1054f4e34259dca51e8bdb42b33bbd
Relates-To:#148
2020-04-07 14:07:45 +00:00
Drew Walters
263f2ecd6f Add remote system shutdown command
This change introduces a system shutdown command that can be used to
power off remote hosts.

Relates-To: #5

Change-Id: I4fa52db055dcb17e344283fbdd772506e12f77fe
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-04-07 13:21:27 +00:00
Stanislav Egorov
f7a2e33b8b Fixed TODO about error definitions
Use SelectOne to replace Select and additional checks.

Change-Id: I40b4780d62fa4b3a0e01cc54ef7dbe4b32a4763d
2020-04-06 11:23:42 -07:00
Drew Walters
a4001d255f Add remote host reboot command
This change introduces a reboot command that allows the user to reboot a
remote host. The reboot command supports any airshipctl remote client
and simply uses the one defined in the remote options.

Relates #5

Change-Id: I6e11430aa60a09172eb3ac0f44d1a6ff17ca91b0
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-04-06 09:26:57 -05:00
Drew Walters
0c3eefe036 Add remote host power status command
This change introduces a command that allows the user to retrieve the
power status of a remote host.

Relates-To: #5

Change-Id: I4d3ded6667a5427ad6814c3d000da3becaec50a1
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-04-03 21:18:18 +00:00
Drew Walters
d25c22a538 Add remote command
This change introduces the remote command. The remote command will be
used to control the power of remote nodes and eventually include the
"bootstrap remotedirect" functionality.

This change is merely an outline of the command. No functionality is
included in this change.

Closes #117

Change-Id: I3291405f36d6a7acc5c8c1f07bd90ad9c2986cd3
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-04-03 20:52:22 +00:00
Drew Walters
b4b792a04f Fix function name misspelling
Change-Id: I8ffb5fc459360fb9cf4cb84dc0d6ab0aa4f30d21
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-04-03 20:49:17 +00:00
Drew Walters
a15f978cad Use new Redfish client
The remote package in airshipctl is tightly coupled to redfish. In the
future, we may need to introduce IPMI or SMASH; however, adding those
clients now would be difficult because of our tight dependence on
redfish. This change implements usage of the new Redfish client, which
implements the generic remote client. It also separates remoteDirect
functionality from the redfish package in order to make it more loosely
coupled.

Relates #5
Closes #122

Change-Id: I45d4ea6e2a4146ea519e94ea701a3ad527e50ca0
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-04-03 18:10:53 +00:00
Drew Walters
bca0a96603 Add mock Redfish client
Remote direct tests mock the go-redfish client in order to avoid making
Redfish API calls. We should be able to mock airshipctl's internal
Redfish API client for unit tests with less side effects. This change
adds a mock Redfish client to reduce the side effects in our remote
direct unit tests.

Relates #5, #122

Change-Id: Ic968c5eefa90aa2065a3f15bb9880edbacf73ece
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-04-03 18:01:45 +00:00
Matt McEuen
e690a4c453 Update Kustomize to v3.3.1
This updates the Kustomize module dependency to v3.3.1,
which has replacement functionality required by the Replacement Transformer
plugin, among other enhancements.

Change-Id: I058114c398464394cfb6b2626cb475d763c1bacc
2020-04-03 12:11:33 -05:00
Zuul
ad8c87611b Merge "Use 16GB flavor for gate job and let it vote" 2020-04-03 17:09:27 +00:00
Zuul
ddfb6ebdbe Merge "Add a HTTPS-->HTTP reverse proxy in front of redfish-emulator" 2020-04-03 15:03:51 +00:00
Zuul
01ca21754d Merge "Fix ironic deployment template" 2020-04-03 14:59:02 +00:00
Zuul
effc58d511 Merge "Add Bare Metal Operator Ironic entrypoints" 2020-04-03 14:46:59 +00:00
Zuul
004e9c47ac Merge "Switch gate to official isogen image" 2020-04-03 08:07:42 +00:00
Zuul
a200af5810 Merge "Remove external role dependency from gate job" 2020-04-03 00:42:01 +00:00
Stanislav Egorov
3063e207b3 Use 16GB flavor for gate job and let it vote
Gate job can be run on a 16GB flavor as well as 32GB.
It's better to use less resources if this works for tests.

Depends-On: https://review.opendev.org/#/c/715055/

Change-Id: I198d640584e80dc6ca3626a6b23855c2aba6bf60
2020-04-02 21:28:34 +00:00
Stanislav Egorov
b4a4be2a7f Switch gate to official isogen image
Change-Id: I65e02251b2f6c747c72a38bb20caf2dd3978ea7c
2020-04-02 14:17:15 -07:00
Vamsi Savaram
5bf96b27d6 Add a HTTPS-->HTTP reverse proxy in front of redfish-emulator
This patch introduces a reverse proxy that terminates an HTTPS
connection and forwards it to the redfish emulator behind that
is running HTTP. Also the reverse proxy presents a self-signed
certificate.

Closes: #136
Relates-To: #136

Change-Id: If6ee705247ae8866d2674bff1ff034277f9c9177
2020-04-02 10:21:37 -05:00
Drew Walters
f3262bc015 Retrieve actual virtual media ID and types
The "GetVirtualID" method is stubbed for our Redifsh implementation and
always returns the same values. This change uses the Redfish client to
retrieve the correct virtual media ID and type.

Closes #141

Change-Id: I319b89c13349e7b2f5bf6eb8ced97c949e5f44b5
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-04-02 15:13:41 +00:00
Ian Howell
bc949f259f Improve error messages related to Selectors
This change modifies the error messages related to Selectors by
delegating formatting to the Selector's String method.

Change-Id: I88acb70751509e588139ea26a8388661b3f63511
Relates-To: #145
2020-04-02 08:07:05 -05:00
Zuul
0ad22d69b6 Merge "Add remote Client interface" 2020-04-02 07:12:28 +00:00
Zuul
17be5ca472 Merge "Remove personal keys from ephemeral userData" 2020-04-01 21:31:33 +00:00
Zuul
5f1eff7858 Merge "Reduce the complexity of ScreenRedfishError" 2020-04-01 21:31:31 +00:00
Zuul
99189e3d5a Merge "Fix Redfish error handling logic" 2020-04-01 21:31:30 +00:00
Zuul
c6a02fc64b Merge "Remove mac address from networkdata" 2020-04-01 21:20:30 +00:00
Zuul
d9c38616e3 Merge "Improve document.Selector's String method" 2020-04-01 21:20:29 +00:00
Ian Howell
68ec25db22 Reduce the complexity of ScreenRedfishError
Change-Id: I69b7e546cd814f5e3b6b890348f4e808058ae6ef
Relates-To: #142
2020-04-01 15:34:51 -05:00