Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Network: support down NIC #130

Merged
merged 23 commits into from
Mar 2, 2022
Merged

Conversation

FingerLeader
Copy link
Member

@FingerLeader FingerLeader commented Feb 17, 2022

Notice: The parameter duration is set to mandatory, because disabling the remote server's NIC is dangerous.

Signed-off-by: FingerLeader <wanxfinger@gmail.com>
Signed-off-by: FingerLeader <wanxfinger@gmail.com>
Signed-off-by: FingerLeader <wanxfinger@gmail.com>
Signed-off-by: FingerLeader <wanxfinger@gmail.com>
Signed-off-by: FingerLeader <wanxfinger@gmail.com>
Signed-off-by: FingerLeader <wanxfinger@gmail.com>
Signed-off-by: FingerLeader <wanxfinger@gmail.com>
Signed-off-by: FingerLeader <wanxfinger@gmail.com>
Signed-off-by: FingerLeader <wanxfinger@gmail.com>
Signed-off-by: FingerLeader <wanxfinger@gmail.com>
Signed-off-by: FingerLeader <wanxfinger@gmail.com>
Signed-off-by: FingerLeader <wanxfinger@gmail.com>
Signed-off-by: FingerLeader <wanxfinger@gmail.com>
@ti-chi-bot
Copy link
Member

ti-chi-bot commented Feb 17, 2022

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • WangXiangUSTC
  • cwen0

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

Signed-off-by: FingerLeader <wanxfinger@gmail.com>
FingerLeader and others added 2 commits February 18, 2022 19:30
Signed-off-by: FingerLeader <wanxfinger@gmail.com>
NICDownCommand := fmt.Sprintf("ifconfig %s down", attack.Device)

cmd := exec.Command("bash", "-c", NICDownCommand)
if err = cmd.Start(); err != nil {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I run it under a non-root user, it runs failed but did not report any error here. Because it will not wait for the command to complete.
https://pkg.go.dev/os/exec#Cmd.Start
It's better to use cmd.CombinedOutput.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But using cmd.CombinedOutput or cmd.Run will make process stalled when exectuing sleep $duration, is it acceptable?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's OK. chaosd should't exit before recover it

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If chaosd will not exit, I think nohup script is useless. Should I remove it?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, we can remove the nohup in cmd

return nil
}

func (s *Server) recoverNICDownScheduled(attack *core.NetworkCommand) error {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this way, we need to set the experiment's status to Destroyed.

I think it's better to refine the schedule framework, and let it recover automatically by the schedule framework.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can refine the schedule framework in next pr

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this solution is temporary, real schedule is need, especially for some dangerous experiments.

Signed-off-by: FingerLeader <wanxfinger@gmail.com>
Signed-off-by: FingerLeader <wanxfinger@gmail.com>
Copy link
Collaborator

@WangXiangUSTC WangXiangUSTC left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rest LGTM

return errors.WithStack(err)
}

if attack.Duration != "-1" {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this should be len(attack.Duration) != 0?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

len(attack.Duration) has been checked before this function, -1 means attack will run continuously. It will be marked in doc.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not check this field before executing this attack?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This field has been checked for necessity before executing, if len(n.Duration) == 0 { return errors.New("duration is required") }, the judgement here is only to decide whether to run continuously.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remember you add a note that the attack.Duration is required, but if users set it to -1, this attack won't be recovered automatically. This behavior is very dangerous. Can we forbid users to set it to -1?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, this -1 option is added on purpose, because I think some user may need this option to let the experiment run continuously, especially those who run NIC down locally rather than remotely. And attack.Duration is hardly ever set to negative numbers, I think -1 is enough to be a warning sign (maybe -2333 or -32768 is better).

Copy link
Collaborator

@WangXiangUSTC WangXiangUSTC left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

recoverCmd := exec.Command("bash", "-c", NICUpCommand)
stdout, err := recoverCmd.CombinedOutput()
if err != nil {
log.Error(recoverCmd.String()+string(stdout), zap.Error(err))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed these log.Error, PTAL : )

Signed-off-by: FingerLeader <wanxfinger@gmail.com>
if err != nil {
return errors.WithStack(err)
}
attack.IPAddress = strings.TrimRight(string(stdoutBytes), "\n\x00")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add some comments to explain why need to trim \x00

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DONE IT!

Signed-off-by: FingerLeader <wanxfinger@gmail.com>
Signed-off-by: FingerLeader <wanxfinger@gmail.com>
Signed-off-by: FingerLeader <wanxfinger@gmail.com>
Copy link
Member

@cwen0 cwen0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@cwen0
Copy link
Member

cwen0 commented Mar 2, 2022

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 2af752c

@ti-chi-bot ti-chi-bot merged commit 81dc95a into chaos-mesh:main Mar 2, 2022
@FingerLeader FingerLeader deleted the add-down-nic branch March 5, 2022 13:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants