When going to a file from diff view some settings overwrite

[AlejandroSanchez90]

Description

When I go to the diff view (diff this), hover over a file, and use gf (go to file), I can see the file, but a lot of my config stops working. For example, :set number gets disabled, and my keys to go down and up (c-n and c-p) stop working, which key stops working () if I do :verbose set number? I get this response

Last set from /nvim/lazy/neogit/lua/neogit/lib/buffer.lua line 413 

Neovim version

nvim v0.10.1

Operating system and version

macOs

Steps to reproduce

1. :NeoGit
2. dd on a branch
3. hover any file
4. press gf
5. on the file try some common keys

Expected behavior

No response

Actual behavior

some config gets overwritten

Minimal config

return {
	"NeogitOrg/neogit",
	event = "VeryLazy",
	dependencies = {
		"nvim-lua/plenary.nvim", -- required
		"sindrets/diffview.nvim", -- optional - Diff integration

		-- Only one of these is needed, not both.
		"ibhagwan/fzf-lua",
	},
	config = function()
		local neogit = require("neogit")
		neogit.setup({})

		local keymap = vim.keymap -- for conciseness
		keymap.set("n", "<leader>go", "<cmd>Neogit<cr>", { desc = "Open NeoGit" })
		keymap.set("n", "<leader>gf", "<cmd>Neogit fetch<cr>", { desc = "Fetch" })
		keymap.set("n", "<leader>gb", "<cmd>Neogit branch<cr>", { desc = "Branch" })
		keymap.set("n", "<leader>gg", "<cmd>Neogit commit<cr>", { desc = "Commit" })
		keymap.set("n", "<leader>gp", "<cmd>Neogit push<cr>", { desc = "Push" })
		keymap.set("n", "<leader>gs", "<cmd>FzfLua git_status<cr>", { desc = "Status" })

		keymap.set("n", "<leader>gdh", "<cmd>DiffviewFileHistory % --base=LOCAL<cr>", { desc = "File github history" })

		--DiffviewFileHistory % --base=LOCAL
	end,
}