Join the Kedro community

M
M
M
D
M
Members
Galen Seilis
G
Galen Seilis
Offline, last seen 17 hours ago
Joined September 19, 2024

I have a question about the memory dataset's default copy method. I noticed that if the data is a pandas dataframe or a numpy array that copy rather than assignment (i.e. making a reference) is used by default. I'm wondering what the rationale for that is. Often making a reference is cheaper in terms of runtime than making either a shallow or deep copy. Why is assignment not the top priority default?

https://docs.kedro.org/en/stable/_modules/kedro/io/memory_dataset.html#MemoryDataset

8 comments
D
G
Y

I am getting a warning when I run pytest on a dummy project:

PytestDeprecationWarning: The hookimpl CovPlugin.pytest_configure_node uses old-style configuration options (marks or attributes).
  Please use the pytest.hookimpl(optionalhook=True) decorator instead
   to configure the hooks.
   See <a target="_blank" rel="noopener noreferrer" href="https://docs.pytest.org/en/latest/deprecations.html#configuring-hook-specs-impls-using-markers">https://docs.pytest.org/en/latest/deprecations.html#configuring-hook-specs-impls-using-markers</a>
    def pytest_configure_node(self, node):

..\venv\lib\site-packages\pytest_cov\plugin.py:265
\venv\lib\site-packages\pytest_cov\plugin.py:265: PytestDeprecationWarning: The hookimpl CovPlugin.pytest_testnodedown uses old-style configuration options (marks or attributes).
  Please use the pytest.hookimpl(optionalhook=True) decorator instead
   to configure the hooks.
   See <a target="_blank" rel="noopener noreferrer" href="https://docs.pytest.org/en/latest/deprecations.html#configuring-hook-specs-impls-using-markers">https://docs.pytest.org/en/latest/deprecations.html#configuring-hook-specs-impls-using-markers</a>
    def pytest_testnodedown(self, node, error):

-- Docs: <a target="_blank" rel="noopener noreferrer" href="https://docs.pytest.org/en/stable/how-to/capture-warnings.html">https://docs.pytest.org/en/stable/how-to/capture-warnings.html</a>

Is this something I should do anything about, or will it be addressed in a future version of Kedro?

6 comments
d
N
G

I am getting some kind of environmental variable or config issue. The module for the project cannot be found. At first I thought it was just one project, but it seems to be something broader.

On my system even creating a fresh project gives the same error.

  1. I create a venv, and activate it.
  2. Install kedro (pip install kedro), which is currently giving 0.19.9
  3. Initialize new kedro project.
  4. Run pytest (which gives a module not found error for the very project I just created).

Troubleshooting advice would be appreciated.

19 comments
J
G
N

Is there a recommended way to run type checkers (e.g. MyPy) on Kedro projects?

2 comments
D
M