Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add multi node (validator) testnet #4377

Open
wants to merge 22 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
- [#4300](https://github.com/ignite/cli/pull/4300) Only panics the module in the most top function level
- [#4327](https://github.com/ignite/cli/pull/4327) Use the TxConfig from simState instead create a new one
- [#4326](https://github.com/ignite/cli/pull/4326) fAdd `buf.build` version to `ignite version` command
- [#4377](https://github.com/ignite/cli/pull/4377) Add multi node (validator) testnet.
- [#4362](https://github.com/ignite/cli/pull/4362) Scaffold `Makefile`

### Changes
Expand Down
56 changes: 55 additions & 1 deletion docs/docs/03-CLI-Commands/01-cli-commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -3666,7 +3666,7 @@ Start a testnet local

**Synopsis**

The commands in this namespace allow you to start your local testnet for development purposes. Currently there is only one feature to create a testnet from any state network (including mainnet).
The commands in this namespace allow you to start your local testnet for development purposes.


The "in-place" command is used to create and start a testnet from current local net state(including mainnet).
Expand All @@ -3675,9 +3675,12 @@ We can create a testnet from the local network state and mint additional coins f

During development, in-place allows you to quickly reboot the chain from a multi-node network state to a node you have full control over.

The "multi-node" initialization and start command is used to set up and launch a multi-node network, allowing you to enable, disable, and providing full interaction capabilities with the chain. The stake amount for each validator is defined in the config.yml file.

**SEE ALSO**

* [ignite testnet in-place](#ignite-testnet-in-place) - Create and start a testnet from current local net state
* [ignite testnet multi-node](#ignite-testnet-multi-node) - Initialize and provide multi-node on/off functionality


## ignite testnet in-place
Expand Down Expand Up @@ -3725,6 +3728,57 @@ ignite chain debug [flags]
-c, --config string path to Ignite config file (default: ./config.yml)
```

## ignite testnet multi-node

Initialize and start multiple nodes

**Synopsis**

The "multi-node" command allows developers to easily set up, initialize, and manage multiple nodes for a testnet environment. This command provides full flexibility in enabling or disabling each node as desired, making it a powerful tool for simulating a multi-node blockchain network during development.

By using the config.yml file, you can define validators with custom bonded amounts, giving you control over how each node participates in the network:

```
validators:
- name: alice
bonded: 100000000stake
- name: validator1
bonded: 100000000stake
- name: validator2
bonded: 200000000stake
- name: validator3
bonded: 300000000stake

```

Each validator's bonded stake can be adjusted according to your testing needs, providing a realistic environment to simulate various scenarios.

The multi-node command not only initializes these nodes but also gives you control over starting, stopping individual nodes. This level of control ensures you can test and iterate rapidly without needing to reinitialize the entire network each time a change is made. This makes it ideal for experimenting with validator behavior, network dynamics, and the impact of various configurations.

All initialized nodes will be stored under the `.ignite/local-chains/<appd>/testnet/` directory, which allows easy access and management.


Usage

```
ignite testnet multi-node [flags]
```

**Options**

```
-r, --reset-once reset the app state once on init
--node-dir-prefix dir prefix for node (default "validator")
-h, --help help for debug
-p, --path string path of the app (default ".")
```

**Options inherited from parent commands**

```
-c, --config string path to Ignite config file (default: ./config.yml)
```

**SEE ALSO**

* [ignite](#ignite) - Ignite CLI offers everything you need to scaffold, test, build, start testnet and launch your blockchain
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ import (
tea "github.com/charmbracelet/bubbletea"
"github.com/stretchr/testify/require"

cmdmodel "github.com/ignite/cli/v29/ignite/cmd/model"
"github.com/ignite/cli/v29/ignite/cmd/model/testdata"
cmdmodel "github.com/ignite/cli/v29/ignite/cmd/bubblemodel"
"github.com/ignite/cli/v29/ignite/cmd/bubblemodel/testdata"
"github.com/ignite/cli/v29/ignite/pkg/cliui/colors"
"github.com/ignite/cli/v29/ignite/pkg/cliui/icons"
cliuimodel "github.com/ignite/cli/v29/ignite/pkg/cliui/model"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ import (
tea "github.com/charmbracelet/bubbletea"
"github.com/stretchr/testify/require"

cmdmodel "github.com/ignite/cli/v29/ignite/cmd/model"
"github.com/ignite/cli/v29/ignite/cmd/model/testdata"
cmdmodel "github.com/ignite/cli/v29/ignite/cmd/bubblemodel"
"github.com/ignite/cli/v29/ignite/cmd/bubblemodel/testdata"
"github.com/ignite/cli/v29/ignite/pkg/cliui/colors"
"github.com/ignite/cli/v29/ignite/pkg/cliui/icons"
cliuimodel "github.com/ignite/cli/v29/ignite/pkg/cliui/model"
Expand Down
200 changes: 200 additions & 0 deletions ignite/cmd/bubblemodel/testnet_multi_node.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,200 @@
package cmdmodel

import (
"bufio"
"context"
"fmt"
"os/exec"
"path/filepath"
"strconv"
"syscall"

tea "github.com/charmbracelet/bubbletea"

"github.com/ignite/cli/v29/ignite/services/chain"
)

type NodeStatus int

const (
Stopped NodeStatus = iota
Running
)

type MultiNode struct {
ctx context.Context
appd string
args chain.MultiNodeArgs

nodeStatuses []NodeStatus
pids []int // Store the PIDs of the running processes
numNodes int // Number of nodes
logs [][]string // Store logs for each node
}

type ToggleNodeMsg struct {
nodeIdx int
}

type UpdateStatusMsg struct {
nodeIdx int
status NodeStatus
}

type UpdateLogsMsg struct{}

func UpdateDeemon() tea.Cmd {
return func() tea.Msg {
return UpdateLogsMsg{}
}
}

// Initialize the model.
func NewModel(ctx context.Context, chainname string, args chain.MultiNodeArgs) MultiNode {
numNodes, err := strconv.Atoi(args.NumValidator)
if err != nil {
panic(err)
}
return MultiNode{
ctx: ctx,
appd: chainname + "d",
args: args,
nodeStatuses: make([]NodeStatus, numNodes), // initial states of nodes
pids: make([]int, numNodes),
numNodes: numNodes,
logs: make([][]string, numNodes), // Initialize logs for each node
}
}

// Implement the Update function.
func (m MultiNode) Init() tea.Cmd {
return nil
}

// ToggleNode toggles the state of a node.
func ToggleNode(nodeIdx int) tea.Cmd {
return func() tea.Msg {
return ToggleNodeMsg{nodeIdx: nodeIdx}
}
}

// Run or stop the node based on its status.
func RunNode(nodeIdx int, start bool, m MultiNode) tea.Cmd {
var (
pid = &m.pids[nodeIdx]
args = m.args
appd = m.appd
)

return func() tea.Msg {
if start {
nodeHome := filepath.Join(args.OutputDir, args.NodeDirPrefix+strconv.Itoa(nodeIdx))
// Create the command to run in background as a daemon
cmd := exec.Command(appd, "start", "--home", nodeHome)

// Start the process as a daemon
cmd.SysProcAttr = &syscall.SysProcAttr{
Setpgid: true, // Ensure it runs in a new process group
}

stdout, err := cmd.StdoutPipe() // Get stdout for logging
if err != nil {
fmt.Printf("Failed to start node %d: %v\n", nodeIdx+1, err)
return UpdateStatusMsg{nodeIdx: nodeIdx, status: Stopped}
}

err = cmd.Start() // Start the node in the background
if err != nil {
fmt.Printf("Failed to start node %d: %v\n", nodeIdx+1, err)
return UpdateStatusMsg{nodeIdx: nodeIdx, status: Stopped}
}

*pid = cmd.Process.Pid // Store the PID
go func() {
scanner := bufio.NewScanner(stdout)
for scanner.Scan() {
line := scanner.Text()
// Add log line to the respective node's log slice
m.logs[nodeIdx] = append(m.logs[nodeIdx], line)
// Keep only the last 5 lines
if len(m.logs[nodeIdx]) > 5 {
m.logs[nodeIdx] = m.logs[nodeIdx][len(m.logs[nodeIdx])-5:]
}
}
}()
return UpdateStatusMsg{nodeIdx: nodeIdx, status: Running}
}
// Use kill to stop the node process by PID
if *pid != 0 {
err := syscall.Kill(-*pid, syscall.SIGTERM) // Stop the daemon process
if err != nil {
fmt.Printf("Failed to stop node %d: %v\n", nodeIdx+1, err)
} else {
*pid = 0 // Reset PID after stopping
}
}
return UpdateStatusMsg{nodeIdx: nodeIdx, status: Stopped}
}
}

// Stop all nodes.
func (m *MultiNode) StopAllNodes() {
for i := 0; i < m.numNodes; i++ {
if m.nodeStatuses[i] == Running {
RunNode(i, false, *m)() // Stop node
}
}
}

// Update handles messages and updates the model.
func (m MultiNode) Update(msg tea.Msg) (tea.Model, tea.Cmd) {
switch msg := msg.(type) {
case tea.KeyMsg:
switch msg.String() {
case "q":
likesToEatFish marked this conversation as resolved.
Show resolved Hide resolved
m.StopAllNodes() // Stop all nodes before quitting
return m, tea.Quit
default:
// Check for numbers from 1 to numNodes
for i := 0; i < m.numNodes; i++ {
if msg.String() == fmt.Sprintf("%d", i+1) {
return m, ToggleNode(i)
}
}
}

case ToggleNodeMsg:
if m.nodeStatuses[msg.nodeIdx] == Running {
return m, RunNode(msg.nodeIdx, false, m) // Stop node
}
return m, RunNode(msg.nodeIdx, true, m) // Start node

case UpdateStatusMsg:
m.nodeStatuses[msg.nodeIdx] = msg.status
return m, UpdateDeemon()
case UpdateLogsMsg:
return m, UpdateDeemon()
}

return m, nil
}

// View renders the interface.
func (m MultiNode) View() string {
output := "Node Control:\n"
for i := 0; i < m.numNodes; i++ {
status := "[Stopped]"
if m.nodeStatuses[i] == Running {
status = "[Running]"
}
output += fmt.Sprintf("%d. Node %d %s --node tcp://127.0.0.1:%d:\n", i+1, i+1, status, 26657-3*i)
output += " [\n"
for _, line := range m.logs[i] {
output += " " + line + "\n"
}
output += " ]\n\n"
}

output += "Press q to quit.\n"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add a sort of colored status bar (https://github.com/charmbracelet/lipgloss) that would let you quit a specific node

return output
}
2 changes: 1 addition & 1 deletion ignite/cmd/chain_debug.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import (
tea "github.com/charmbracelet/bubbletea"
"github.com/spf13/cobra"

cmdmodel "github.com/ignite/cli/v29/ignite/cmd/model"
cmdmodel "github.com/ignite/cli/v29/ignite/cmd/bubblemodel"
chainconfig "github.com/ignite/cli/v29/ignite/config/chain"
"github.com/ignite/cli/v29/ignite/pkg/chaincmd"
"github.com/ignite/cli/v29/ignite/pkg/cliui"
Expand Down
2 changes: 1 addition & 1 deletion ignite/cmd/chain_serve.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import (
tea "github.com/charmbracelet/bubbletea"
"github.com/spf13/cobra"

cmdmodel "github.com/ignite/cli/v29/ignite/cmd/model"
cmdmodel "github.com/ignite/cli/v29/ignite/cmd/bubblemodel"
"github.com/ignite/cli/v29/ignite/pkg/cliui"
uilog "github.com/ignite/cli/v29/ignite/pkg/cliui/log"
cliuimodel "github.com/ignite/cli/v29/ignite/pkg/cliui/model"
Expand Down
1 change: 1 addition & 0 deletions ignite/cmd/testnet.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ func NewTestnet() *cobra.Command {

c.AddCommand(
NewTestnetInPlace(),
NewTestnetMultiNode(),
)

return c
Expand Down
Loading
Loading